Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Once RAP4 is running, there are maintenance tasks you might want to do. The most frequently used tasks are described here for RAP-3 maintainers to refer to.
The purpose of maintaining software is to guarantee continuous operation for all users. In this chapter we use the RAP4 server at OUNL as an example, trusting that you will figure out what to do for servers of your own.
Here is an overview of all tasks described on this page. Refer to the related section below for details on each specific task
Connect
Before doing any maintenance, you need credentials to gain access to your server. Then you can connect to the server. We assume you gain access to a command-line interface (CLI) of your server. In this text that CLI is /bin/bash
on a linux machine.
Check the configuration When you start a maintenance session, you may want to check on the system. RAP4 runs in containers on a docker-platform, so you can check whether the containers are running and you can check the configuration in which they should be running.
Upgrade to a new version of RAP
Upgrade to a new version of Ubuntu
Refresh the configuration Refreshing the RAP4 configuration is something you need to do only when developers tell you to do that.
The RAP-server has been configured to communicate via ssh
. The Open Universiteit allows ssh-connections only through VPN. So I made sure my VPN-connection is active.
I entered the server using command ssh rap.cs.ou.nl
and the right user/password combination, upon which I gained access to the CLI.
The RAP4-instance is installed from directory ~\RAP4
, which is the working directory from which maintenance is done.
This is what you can expect to see:
The server should show at least two containers, a database container called rap4_db
and a RAP-container called rap4
. There may be a third container called phpmyadmin
, which is there to gain access to the database (for maintainers only). To verify, give the command docker ps
. This is what you may expect to see:
In this case it appears that a fourth container is running. This poses no problem, because other processes may run concurrently without interfering with RAP4.
The configuration file that specifies this configuration is the only file in the working directory:
By inspecting the contents you can see whether the configuration matches what you see in docker
. This is what you can expect in docker-compose.yml
:
The directory volumes
contains the Ampersand data, which is kept outside the containers so the data persists if containers get killed or if you restart the software.
There are three data sets. The directory log
contains logging information of the RAP4-server. The directory mysql
contains the data from MariaDB. The directory scripts
contains the student scripts and the files they generated when working in RAP4.
If you need to upgrade RAP4 to the latest release run this command:
As you can see in the configuration (docker-compose.yml
)
When you connect to the server, you get hints about the state your Ubuntu server is in. E.g.
In such cases you can update by giving two commands:
Sometimes a package is kept back because there is a problem with dependencies. You will have to upgrade such packages by hand.
In the rare event that the configuration of RAP4 has changed (to be announced by the developers), you must update the file docker-compose.yml
by hand, using the command:
If you want to deploy RAP4 on your own server, you might try to copy how I deployed RAP4 at the OUNL. This chapter tells you how.
In early 2016 the need arose to replace the RAP2 implementation of Ampersand by a RAP3 implementation, because RAP2 was insufficiently maintainable. This environment is used by students for completing the course Rule Based Design (OBR, code IM0103). This implementation is hosted by ICTS, the IT-department of the university.
This chapter is an account of the installation process. It serves the following purposes:
It is an example for others who want to deploy Ampersand. We get requests now and then by people who want to deploy Ampersand, so we figured it is nice to have a documented example for them.
It documents the installation we made for the Open University. We want maintenance of RAP4 to be transferable to other persons, so we need to document the choices made and the reasons for making them.
It contains all information needed to make a deployment script for automated deployment. We have automated the deployment with Docker, so that RAP4 will always be up to date with the most recent stable release of Ampersand.
Each step in the installation process gets a separate section in this text.
I got a server from the Open University's IT-department.
The following settings apply:
At the OUNL, we need VPN to gain access with SSH to a server. This requires approval from the IT department. I got a raw Ubuntu machine, meaning that the port settings (specified above) and VPN have to be requested at the IT-servicedesk.
I can now access this machine through SSH (using PUTTY, which I downloaded from the Internet), but only after installing a VPN-tunnel to the server (using Pulse Secure). In the sequel, I will refer to this machine as "the server". This gave me access through a command line interface (CLI). Ubuntu gave me bash as its CLI.
Since this is a fresh machine, docker has to be installed. By just typing docker
, the server advised to install Docker by means of the command sudo apt install docker.io
. This turned out to be a bad advice, because it resulted in a too old version of docker. Instead, I followed the instructions on https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/
for installing docker. The instructions for docker-compose are found on https://docs.docker.com/compose/install/
.
Then I checked that everything went successfully by means of the which
-command:
We need only one file: docker-compose.yml
To get it, I used the wget
command, which gets stuff from the web:
To install RAP4:
To check whether this worked, I went to my browser and navigated to http://145.20.188.96/RAP4
.
It took a while to get started, because it was building a fresh database.
I checked whether the containers are running by means of the docker ps
command.
Completion of this step allowed access to RAP4 from an arbitrary computer on the internet:
The database is accessible on port 8080:
If you want to deploy RAP4 to your laptop, this chapter tells you how.
This example has worked on my Macbook. If you have a different computer, you may either change the commands below to the commands of your own computer or set up a virtual Linux machine on your laptop.
You need docker. Follow the instructions onhttps://docs.docker.com/.
You need git. Follow the instructions on https://github.com/git-guides/install-git
.
You need to use a command-line, you need permission to install software on your laptop and you need to be connected to the Internet.
Start in your command-line interface (CLI, terminal on a macbook) and go to (or make) a working directory from which you want to keep Git repositories.
Import the required files from Github (note: you may need to log in into Github):
Expect roughly this response:
Now go to the directory from which to install RAP:
You need to create a proxy network to let RAP communicate with your browser (no problem if it already exists; docker will tell you with an appropriate message)
When docker has made the network, it tells you something like this:
Now spin up RAP4 (note: in older versions of docker use docker-compose
instead of docker compose
. The difference is just a hyphen):
The first time you spin up RAP4, docker will spontaneously download the images it needs. That generates substantial output. At the end docker will show the containers that have been started:
Now, you need to "climb into" the rap4 container to execute one command manually:
You now get a prompt like root@bc774265c8ef:/usr/local/project#
that indicates that you are inside the rap4 container. Once inside, you need to give the following chmod
command to enable RAP to generate prototypes.
Finally, type exit to leave the rap4-container and return to the CLI of your laptop.
Now you are done. Just check whether everything works:
Type localhost
in the URL-field of your browser, to see if RAP4 has started. You should see this:
You will find that the database is accessible on http://phpmyadmin.localhost
The demonstration application, Enrollment, is accessible on http://enroll.localhost
Note that RAP runs on the insecure http://
instead of https://
. This is not a problem if you keep your laptop safe from outsiders' trespassing. If you , you need a secure setup.
server name
lnx-hrl-202v
OS
Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-153-generic x86_64
Admin user name
sjo
verification type
password (Stef Joosten knows the password)
Size
2 core, 7GB
Inbound port: RAP4 (HTTP)
TCP/80
Inbound port: HTTPS
TCP/443
Inbound port: SSH
TCP/22
Public IP-adres
145.20.188.96
domain name for calling the application
internet access needed
for generating prototypes because the prototype generator draws from the Ampersand-github repository.
RAP is a Repository for Ampersand Projects. This tool is being used by the Open University of the Netherlands in the course Rule-Based Design. It lets users analyse Ampersand models, generate functional specifications, and make prototypes of information systems. It is the primary tool for students in relation to Ampersand.
Deployment means to install RAP4 on a web server or a laptop of your own. The easiest way to try Ampersand, however, is by just using the OUNL-installation and avoid the installation.
We use Docker for automating the deployment and making RAP portable over different platforms. The following three sections report three different installations we did:
to deploy RAP on an Ordina server in the Azure cloud;
to deploy RAP at the Open University on a server in the OUNL-datacenter.
These three reports will most likely contain enough information to let you reproduce the installation on a computer or platform of your own.
The last section of this chapter discusses the making of docker images, which you may need if the above does not work because the images are (for any reason whatsoever) unavailable on the internet.
Don't hesitate to contact Han Joosten. He'll be glad to help out.
The docker configuration uses volumes to link to the file system of the host computer. As a result the status of RAP will survive containers (i.e. you can start, stop, kill, restart containers without losing data). However, you must ensure write permission for the container on runtime throughout all three subdirectories of the volumes
directory. The pod in which RAP runs defines a volume called log
in which the log files are written and a volume called scripts
in which user scripts are stored. The pod in which MariaDB runs writes its mysql
-directory to a volume called mysql
. This means that the files are stored outside the container on the machine that hosts the docker platform.
You have a host computer in which the docker platform runs. (In deploying RAP4 for the OUNL that would be the machine with domain name rap.cs.ou.nl
). To avoid permission errors (and the use of sudo
), add your user to the docker
group. Read more.
For developing RAP you use slightly different settings. For example you want to reset the database when developing, but this feature must be entirely impossible during production. For this purpose we have two source files: RAP4dev.adl
and RAP4prod.adl
. The only difference between the two is configuration settings. Follow the code to find out.
Docker is sheer magic. You don't have to understand it to run it. However, if you insist on knowing how it is done...
Here are ten steps to install RAP4 manually, without Docker, on a freshly created server:
You need a server that is connected to the internet, because RAP takes updates from a GitHub repository. A 2-core/8GB server is sufficient. A memory size under 3GB has shown to be insufficient. Ports for HTTP, HTTPS, SSL, and SFTP must be open.
You need a database and a web-server. This explains why a LAMP-server is an obvious choice if you use a preconfigured server.
A quick way to install is to copy the source code of RAP4 on your server and compile it with Ampersand. That gives you the RAP4 webapplication. You will find the complete Ampersand source code of RAP4 on https://github.com/AmpersandTarski/RAP. The main file is ./RAP4/RAP4dev.adl
for the development version and ./RAP4/RAP4prod.adl
for the production version.
If you don't have an Ampersand compiler, you can build one using the Haskell sources of that compiler. You will find the source code on the development branch of the Ampersand repository (https://github.com/AmpersandTarski/Ampersand/tree/development). You can verify success by asking the Ampersand for its version:
If you don't have an Ampersand compiler, but docker runs on your machine, you can simply use the most recent ampersand compiler by
Use Git to create a local clone of these repositories. Git is preferred over copying the files, because you can repeatedly use it to ensure you get the right version.
To build an Ampersand compiler, you need Haskell. However, you can also use the latest executable Ampersand-compiler, which is published periodically on http://ampersandtarski.github.io/. This way, you don't need to build Ampersand and you can avoid installing Haskell.
If you want, create your own Ampersand compiler. This way, you can pick a version by which to generate RAP4.
LaTeX and GraphViz are called by RAP4 to generate documentation. Therefore, both must be installed on the server.
You need to generate RAP4 to facilitate regular updates to the system.
Some local settings may apply, which are brought together in one file. Use this to administer things like database account(s) and PHP time limit, logging, etcetera.
Things that are necessary for testing and development, such as logging, can be changed in the production version. These things are usually adapted shortly before going to production.
This chapter discusses the process of baking (i.e. creating) images for docker hub. It also gives a recipe for creating docker images. If you are interested only in installing RAP4, you do not need this chapter.
Docker images are baked when the Ampersand source code for RAP4 is ready to be deployed. This results in three files, to be stored on Docker hub:
RAP4 is deployed as shown in this picture
Knowing what needs to be done allows you to understand how we make Ampersand's docker images. If you just want to do it, follow the steps below. We assume that you are working on an Ubuntu machine with bash
as its command line interface.
First I checked that docker and docker-compose are installed on my computer:
If you need to install docker, follow the instructions on https://docs.docker.com/engine/installation .
RAP is built on Ampersand and is compiled with Ampersand. For this reason, the RAP image builds on an Ampersand image. Run ./build.sh to build the initial ampersand container that serves as a base for the RAP4 application (or any other Ampersand application). This base images holds all required packages and the (at that moment) latest version of the ampersand compiler the workflow around this container can/should be improved since now the easiest way to rebuild is to remove the container (docker rmi ampersand:latest)
I cloned the docker files into a local directory by the following command:
and sat back to watch an image being created. This takes over an hour. I left the session up and running, because stopping the session means that the docker build process stops. After coming back a day later, I verified that the image is present by running the same command again. That produced the following output:
When you build an image where nothing changes (as shown above), this takes virtually no time at all. However, if you run a fresh image from scratch, it takes a while. Should you want to close your SSH-session in which docker is running, that docker process will be killed. If you want to leave docker running in the background, https://www.tecmint.com/keep-remote-ssh-sessions-running-after-disconnection/
will help. Best thing is to anticipate on it. However, if you didn't anticipate this, you can still send docker to the background by ^Z
and give the disown -h
command. (The latter is however the least elegant way.)
I checked whether all images are built by the docker images
command:
For this step, you need an account on docker-hub. Then Docker will simply push the generated images to docker hub by means of the push-command:
At this point the images are published and this chapter is done. However, it is good to discuss a few collateral issues: deployment, maintenance and security.
We are making three containers: one for the database, one for the RAP4 application, and one for PhpMyAdmin. Containers are built from images by the command docker-compose
:
This command brings all three containers up and runs them in the background. For this to happen, the file docker-compose.yml
must be in the current directory.
I checked whether the containers are running by means of the docker ps
command.
Completion of this step allowed access to RAP4 from an arbitrary computer on the internet:
The database is accessible on port 8080:
The docker-compose up
command aggregates the output of each container. When the command exits, all containers are stopped. Running docker-compose up -d
starts the containers in the background and leaves them running.
To interfere with RAP4 as it is running, you need to get into the Rap4 container. It is not enough just being on the server, because all the data is in the container (rather than directly on the server). To go into the Rap4 container, use the command
To inspect and change the local settings, you need the file localsettings.php
on directory ~/git/Ampersand-models/RAP4/include
. This step requires section 5 to be finished successfully. This file contains comments that guide you to use the correct settings in a development situation and in a production situation. Read the file and follow the instructions it contains, especially when making the transition from development to production.
In the source code of RAP4, in the file SIAM_importer.adl:
disable "RAP4_LoginForDevelopment.ifc", to prevent users from seeing
enable "RAP4_LoginForProduction.ifc"
disable "../SIAM/SIAM_AutoLoginAccount.adl"
Is there anything we must alter in localsettings.php before going live?
This is decribed in the file docker-compose.yml
In early 2017 the need arose for an Ampersand implementation in Ordina's cloud, to let young professionals get acquainted with Ampersand. We chose to implement RAP4 in Azure, because Ordina has an Azure subscription. RAP4 is the same environment that our students use at the Open Universiteit, so the maintainance of RAP4 can be reused for both target audiences.
This chapter is an account of the deployment of RAP4. You can use it as an example if you want to do it yourself.
Deployment has been automated using docker, because we want to make the deployment as much as possible automated. The required images are available on Docker-hub, so deployment is immediate. Automated deployment by docker means that we can quickly release updates too.
We use Docker to simplify the deployment of RAP and make this installation portable to many different computers.
We took a simple Linux machine, installed docker, and built docker images for RAP4 and created three containers: db, rap4, and phpmyadmin. The first one, db, runs MariaDB and contains the administration of RAP4 and the Ampersand scripts of users. The rap4 container contains the RAP4 application proper. It is accessed on port 80. The container phpmyadmin is present for maintainers to inspect the database through phpMyAdmin. It is accessible through port 8080.
Each step in the installation process is described by a separate section in the sequel.
I needed an Azure account to enter the Azure portal and install a server for Ampersand. I got my account from Ordina. Azure offers preconfigured installations to kick-start a virtual machine. In principle, any server that runs docker will work. I picked CoreOS, which is a minimal Linux platform and suitable for container computing with docker.
The following settings were made:
I verified completion of this step by checking the Azure dashboard. It shows machine Wolfram
running.
I have verified the machine was live by logging in via Putty (a popular SSH-client). I accessed this machine, using ampersandadmin@137.117.190.28
and the correct password. In the sequel, I will refer to this machine as "the server".
To perform this step, I didn't wait until step1 was completed. It is sufficient that the Netwerkbeveiligingsgroep Wolfram-nsg
exists. In order to access RAP4 from the internet, we told the virtual machine to open port 80 for http and 443 for https. These settings were made in the Azure portal via Resourcegroepen > AmpersandRAP4 > Wolfram-nsg (Netwerkbeveiligingsgroep) > inkomende beveiligingsregels
. A rules was added to make the server listen to port 80/TCP.
No rule was added for the database, so the database is only open to phpMyAdmin and RAP4, or from within its container.
This step requires a server, so you must have finished section 1 successfully. (If docker
and docker-compose
are available, this step is superfluous.) To install docker I followed the following steps in the given order:
Of course, the last step (reboot) requires reconnecting to the machine.
Then I checked that docker is running by:
I checked that docker-compose is available by:
which docker-compose
To check whether ampersandadmin is member of the docker group, use command cat /etc/group.
We need only one file: docker-compose.yml
To get it, I used the wget
command, which gets stuff from the web:
To install RAP4:
To check whether this worked, I went to my browser and navigated to http://137.117.190.28/
.
It took a while to get started, because it was building a fresh database.
I checked whether the containers are running by means of the docker ps
command.
Completion of this step allowed access to RAP4 from an arbitrary computer on the internet:
The database is accessible on port 8080:
subscription in Azure
Ordina TC - RT O Pega - Learning
server name
Wolfram
type VM-disk
SDD
OS
Linux
configuration
Ubuntu 18.04
Admin user name
ampersandadmin
verification type
password (Stef Joosten knows the password)
Resource group (in Azure)
AmpersandRAP4
location (in Azure)
Western Europe
Size
Standard DS11 v2 (2 cores, 14 GB memory)
Inbound port: RAP4 (HTTP)
TCP/80
Inbound port: HTTPS
TCP/443
Inbound port: SSH
TCP/22
Public IP-adres
137.117.190.28 (static)
the full name for calling the application
http://137.117.190.28/RAP4
step
Linux command
To work as root
sudo -i
To install docker, I followed the instructions on https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-18-04.
To install docker-compose
apt-get install docker-compose
To add the current user (ampersandadmin) to the docker group
usermod -aG docker $(whoami)
To start the docker daemon
now reboot the machine
reboot