The purpose of maintaining software is to guarantee continuous operation for all users. In this chapter we use the RAP4 server at OUNL as an example, trusting that you will figure out what to do for servers of your own.

Tasks

Here is an overview of all tasks described on this page. Refer to the related section below for details on each specific task

1. Connect Before doing any maintenance, you need credentials to gain access to your server. Then you can connect to the server. We assume you gain access to a command-line interface (CLI) of your server. In this text that CLI is /bin/bash on a linux machine.

2. Check the configuration When you start a maintenance session, you may want to check on the system. RAP4 runs in containers on a docker-platform, so you can check whether the containers are running and you can check the configuration in which they should be running.

3. Upgrade to a new version of RAP

4. Upgrade to a new version of Ubuntu

5. Refresh the configuration Refreshing the RAP4 configuration is something you need to do only when developers tell you to do that.

Connecting to the RAP4 server

The RAP-server has been configured to communicate via ssh. The Open Universiteit allows ssh-connections only through VPN. So I made sure my VPN-connection is active.

I entered the server using command ssh rap.cs.ou.nl and the right user/password combination, upon which I gained access to the CLI.

The RAP4-instance is installed from directory ~\RAP4, which is the working directory from which maintenance is done.

This is what you can expect to see:

Checking which containers are running

The server should show at least two containers, a database container called rap4_db and a RAP-container called rap4. There may be a third container called phpmyadmin, which is there to gain access to the database (for maintainers only). To verify, give the command docker ps. This is what you may expect to see:

In this case it appears that a fourth container is running. This poses no problem, because other processes may run concurrently without interfering with RAP4.

The configuration file that specifies this configuration is the only file in the working directory:

By inspecting the contents you can see whether the configuration matches what you see in docker. This is what you can expect in docker-compose.yml:

The directory volumes contains the Ampersand data, which is kept outside the containers so the data persists if containers get killed or if you restart the software.

There are three data sets. The directory log contains logging information of the RAP4-server. The directory mysql contains the data from MariaDB. The directory scripts contains the student scripts and the files they generated when working in RAP4.

Upgrade to a new version of RAP

If you need to upgrade RAP4 to the latest release run this command:

As you can see in the configuration (docker-compose.yml)

Upgrade to a new version of Ubuntu

When you connect to the server, you get hints about the state your Ubuntu server is in. E.g.

In such cases you can update by giving two commands:

Sometimes a package is kept back because there is a problem with dependencies. You will have to upgrade such packages by hand.

Refreshing the code

In the rare event that the configuration of RAP4 has changed (to be announced by the developers), you must update the file docker-compose.ymlby hand, using the command:

In early 2017 the need arose for an Ampersand implementation in Ordina's cloud, to let young professionals get acquainted with Ampersand. We chose to implement RAP4 in Azure, because Ordina has an Azure subscription. RAP4 is the same environment that our students use at the Open Universiteit, so the maintainance of RAP4 can be reused for both target audiences.

This chapter is an account of the deployment of RAP4. You can use it as an example if you want to do it yourself.

Deployment has been automated using docker, because we want to make the deployment as much as possible automated. The required images are available on Docker-hub, so deployment is immediate. Automated deployment by docker means that we can quickly release updates too.

0. Deployment approach

We use Docker to simplify the deployment of RAP and make this installation portable to many different computers.

We took a simple Linux machine, installed docker, and built docker images for RAP4 and created three containers: db, rap4, and phpmyadmin. The first one, db, runs MariaDB and contains the administration of RAP4 and the Ampersand scripts of users. The rap4 container contains the RAP4 application proper. It is accessed on port 80. The container phpmyadmin is present for maintainers to inspect the database through phpMyAdmin. It is accessible through port 8080.

Each step in the installation process is described by a separate section in the sequel.

1. Setting up the virtual machine

I needed an Azure account to enter the Azure portal and install a server for Ampersand. I got my account from Ordina. Azure offers preconfigured installations to kick-start a virtual machine. In principle, any server that runs docker will work. I picked CoreOS, which is a minimal Linux platform and suitable for container computing with docker.

The following settings were made:

I verified completion of this step by checking the Azure dashboard. It shows machine Wolfram running.

I have verified the machine was live by logging in via Putty (a popular SSH-client). I accessed this machine, using ampersandadmin@137.117.190.28 and the correct password. In the sequel, I will refer to this machine as "the server".

2. Port settings

To perform this step, I didn't wait until step1 was completed. It is sufficient that the Netwerkbeveiligingsgroep Wolfram-nsg exists. In order to access RAP4 from the internet, we told the virtual machine to open port 80 for http and 443 for https. These settings were made in the Azure portal via Resourcegroepen > AmpersandRAP4 > Wolfram-nsg (Netwerkbeveiligingsgroep) > inkomende beveiligingsregels. A rules was added to make the server listen to port 80/TCP.

No rule was added for the database, so the database is only open to phpMyAdmin and RAP4, or from within its container.

3. Installing Docker

This step requires a server, so you must have finished section 1 successfully. (If docker and docker-compose are available, this step is superfluous.) To install docker I followed the following steps in the given order:

Of course, the last step (reboot) requires reconnecting to the machine.

Then I checked that docker is running by:

I checked that docker-compose is available by:

which docker-compose

To check whether ampersandadmin is member of the docker group, use command cat /etc/group.

4. Obtaining the install instruction

We need only one file: docker-compose.yml To get it, I used the wget command, which gets stuff from the web:

5. Installing RAP4

To install RAP4:

To check whether this worked, I went to my browser and navigated to http://137.117.190.28/. It took a while to get started, because it was building a fresh database.

I checked whether the containers are running by means of the docker ps command.

Completion of this step allowed access to RAP4 from an arbitrary computer on the internet:

The database is accessible on port 8080:

Permissions for log files

The docker configuration uses volumes to link to the file system of the host computer. As a result the status of RAP will survive containers (i.e. you can start, stop, kill, restart containers without losing data). However, you must ensure write permission for the container on runtime throughout all three subdirectories of the volumes directory. The pod in which RAP runs defines a volume called log in which the log files are written and a volume called scripts in which user scripts are stored. The pod in which MariaDB runs writes its mysql-directory to a volume called mysql. This means that the files are stored outside the container on the machine that hosts the docker platform.

Docker group

You have a host computer in which the docker platform runs. (In deploying RAP4 for the OUNL that would be the machine with domain name rap.cs.ou.nl). To avoid permission errors (and the use of sudo), add your user to the docker group. .

Development vs. Production versions

For developing RAP you use slightly different settings. For example you want to reset the database when developing, but this feature must be entirely impossible during production. For this purpose we have two source files: RAP4dev.adl and RAP4prod.adl. The only difference between the two is configuration settings. Follow the code to find out.

If you want to deploy RAP4 on your own server, you might try to copy how I deployed RAP4 at the OUNL. This chapter tells you how.

In early 2016 the need arose to replace the RAP2 implementation of Ampersand by a RAP3 implementation, because RAP2 was insufficiently maintainable. This environment is used by students for completing the course Rule Based Design (OBR, code IM0103). This implementation is hosted by ICTS, the IT-department of the university.

This chapter is an account of the installation process. It serves the following purposes:

1. It is an example for others who want to deploy Ampersand. We get requests now and then by people who want to deploy Ampersand, so we figured it is nice to have a documented example for them.

2. It documents the installation we made for the Open University. We want maintenance of RAP4 to be transferable to other persons, so we need to document the choices made and the reasons for making them.

3. It contains all information needed to make a deployment script for automated deployment. We have automated the deployment with Docker, so that RAP4 will always be up to date with the most recent stable release of Ampersand.

Each step in the installation process gets a separate section in this text.

1. Setting up the virtual machine

I got a server from the Open University's IT-department.

The following settings apply:

Getting access to the server

At the OUNL, we need VPN to gain access with SSH to a server. This requires approval from the IT department. I got a raw Ubuntu machine, meaning that the port settings (specified above) and VPN have to be requested at the IT-servicedesk.

I can now access this machine through SSH (using PUTTY, which I downloaded from the Internet), but only after installing a VPN-tunnel to the server (using Pulse Secure). In the sequel, I will refer to this machine as "the server". This gave me access through a command line interface (CLI). Ubuntu gave me bash as its CLI.

Installing Docker

Since this is a fresh machine, docker has to be installed. By just typing docker, the server advised to install Docker by means of the command sudo apt install docker.io. This turned out to be a bad advice, because it resulted in a too old version of docker. Instead, I followed the instructions on https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/ for installing docker. The instructions for docker-compose are found on https://docs.docker.com/compose/install/ .

Then I checked that everything went successfully by means of the which-command:

Obtaining the files we need

We need only one file: docker-compose.yml To get it, I used the wget command, which gets stuff from the web:

Installing RAP4

To install RAP4:

To check whether this worked, I went to my browser and navigated to http://145.20.188.96/RAP4. It took a while to get started, because it was building a fresh database.

I checked whether the containers are running by means of the docker ps command.

Completion of this step allowed access to RAP4 from an arbitrary computer on the internet:

The database is accessible on port 8080:

Docker images

This chapter discusses the process of baking (i.e. creating) images for docker hub. It also gives a recipe for creating docker images. If you are interested only in installing RAP4, you do not need this chapter.

Docker images are baked when the Ampersand source code for RAP4 is ready to be deployed. This results in three files, to be stored on Docker hub:

The release train

RAP4 is deployed as shown in this picture

A recipe for creating images

Knowing what needs to be done allows you to understand how we make Ampersand's docker images. If you just want to do it, follow the steps below. We assume that you are working on an Ubuntu machine with bash as its command line interface.

Check your Docker installation

First I checked that docker and docker-compose are installed on my computer:

If you need to install docker, follow the instructions on .

4. Building an Ampersand image

RAP is built on Ampersand and is compiled with Ampersand. For this reason, the RAP image builds on an Ampersand image. Run ./build.sh to build the initial ampersand container that serves as a base for the RAP4 application (or any other Ampersand application). This base images holds all required packages and the (at that moment) latest version of the ampersand compiler the workflow around this container can/should be improved since now the easiest way to rebuild is to remove the container (docker rmi ampersand:latest)

4. Making a Docker image

I cloned the docker files into a local directory by the following command:

and sat back to watch an image being created. This takes over an hour. I left the session up and running, because stopping the session means that the docker build process stops. After coming back a day later, I verified that the image is present by running the same command again. That produced the following output:

When you build an image where nothing changes (as shown above), this takes virtually no time at all. However, if you run a fresh image from scratch, it takes a while. Should you want to close your SSH-session in which docker is running, that docker process will be killed. If you want to leave docker running in the background, https://www.tecmint.com/keep-remote-ssh-sessions-running-after-disconnection/ will help. Best thing is to anticipate on it. However, if you didn't anticipate this, you can still send docker to the background by ^Z and give the disown -h command. (The latter is however the least elegant way.)

I checked whether all images are built by the docker images command:

5. Publishing to docker-hub

For this step, you need an account on docker-hub. Then Docker will simply push the generated images to docker hub by means of the push-command:

At this point the images are published and this chapter is done. However, it is good to discuss a few collateral issues: deployment, maintenance and security.

6. Deploying directly on this server

We are making three containers: one for the database, one for the RAP4 application, and one for PhpMyAdmin. Containers are built from images by the command docker-compose:

This command brings all three containers up and runs them in the background. For this to happen, the file docker-compose.yml must be in the current directory.

I checked whether the containers are running by means of the docker ps command.

Completion of this step allowed access to RAP4 from an arbitrary computer on the internet:

The database is accessible on port 8080:

7. Maintenance

The docker-compose up command aggregates the output of each container. When the command exits, all containers are stopped. Running docker-compose up -d starts the containers in the background and leaves them running.

To interfere with RAP4 as it is running, you need to get into the Rap4 container. It is not enough just being on the server, because all the data is in the container (rather than directly on the server). To go into the Rap4 container, use the command

10. Local Settings

To inspect and change the local settings, you need the file localsettings.php on directory ~/git/Ampersand-models/RAP4/include. This step requires section 5 to be finished successfully. This file contains comments that guide you to use the correct settings in a development situation and in a production situation. Read the file and follow the instructions it contains, especially when making the transition from development to production.

11. Last minute changes before going to production

1. In the source code of RAP4, in the file SIAM_importer.adl:

   1. disable "RAP4_LoginForDevelopment.ifc", to prevent users from seeing

   2. enable "RAP4_LoginForProduction.ifc"

This is decribed in the file docker-compose.yml

RAP is a Repository for Ampersand Projects. This tool is being used by the Open University of the Netherlands in the course Rule-Based Design. It lets users analyse Ampersand models, generate functional specifications, and make prototypes of information systems. It is the primary tool for students in relation to Ampersand.

Deployment means to install RAP4 on a web server or a laptop of your own. The easiest way to try Ampersand, however, is by just using the OUNL-installation and avoid the installation.

Automated deployment

We use Docker for automating the deployment and making RAP portable over different platforms. The following three sections report three different installations we did:

• to ;

• to in the Azure cloud;

• to on a server in the OUNL-datacenter.

These three reports will most likely contain enough information to let you reproduce the installation on a computer or platform of your own.

The last section of this chapter discusses the , which you may need if the above does not work because the images are (for any reason whatsoever) unavailable on the internet.

Questions? Need help?

Don't hesitate to contact Han Joosten. He'll be glad to help out.

If you want to deploy RAP4 to your laptop, this chapter tells you how.

This example has worked on my Macbook. If you have a different computer, you may either change the commands below to the commands of your own computer or set up a virtual Linux machine on your laptop.

Prerequisites

You need docker. Follow the instructions onhttps://docs.docker.com/.

You need git. Follow the instructions on https://github.com/git-guides/install-git.

You need to use a command-line, you need permission to install software on your laptop and you need to be connected to the Internet.

Start in your command-line interface (CLI, terminal on a macbook) and go to (or make) a working directory from which you want to keep Git repositories.

Obtaining the files we need

Import the required files from Github (note: you may need to log in into Github):

Expect roughly this response:

Now go to the directory from which to install RAP:

You need to create a proxy network to let RAP communicate with your browser (no problem if it already exists; docker will tell you with an appropriate message)

When docker has made the network, it tells you something like this:

Now spin up RAP4 (note: in older versions of docker use docker-compose instead of docker compose. The difference is just a hyphen):

The first time you spin up RAP4, docker will spontaneously download the images it needs. That generates substantial output. At the end docker will show the containers that have been started:

Now, you need to "climb into" the rap4 container to execute one command manually:

You now get a prompt like root@bc774265c8ef:/usr/local/project# that indicates that you are inside the rap4 container. Once inside, you need to give the following chmod command to enable RAP to generate prototypes.

Finally, type exit to leave the rap4-container and return to the CLI of your laptop.

Now you are done. Just check whether everything works:

Type localhost in the URL-field of your browser, to see if RAP4 has started. You should see this:

Note that RAP runs on the insecure http:// instead of https://. This is not a problem if you keep your laptop safe from outsiders' trespassing. If you , you need a secure setup.

You will find that the database is accessible on http://phpmyadmin.localhost

The demonstration application, Enrollment, is accessible on http://enroll.localhost

What Docker needs to do...

Docker is sheer magic. You don't have to understand it to run it. However, if you insist on knowing how it is done...

Here are ten steps to install RAP4 manually, without Docker, on a freshly created server:

1. Setting up a server

You need a server that is connected to the internet, because RAP takes updates from a GitHub repository. A 2-core/8GB server is sufficient. A memory size under 3GB has shown to be insufficient. Ports for HTTP, HTTPS, SSL, and SFTP must be open.

2. Getting MariaDB (MySQL) and phpMyAdmin to work

You need a database and a web-server. This explains why a LAMP-server is an obvious choice if you use a preconfigured server.

3. Uploading and running RAP4

A quick way to install is to copy the source code of RAP4 on your server and compile it with Ampersand. That gives you the RAP4 webapplication. You will find the complete Ampersand source code of RAP4 on . The main file is ./RAP4/RAP4dev.adl for the development version and ./RAP4/RAP4prod.adl for the production version.

4. Filling the Git repository with Ampersand files and Ampersand models

If you don't have an Ampersand compiler, you can build one using the Haskell sources of that compiler. You will find the source code on the development branch of the Ampersand repository (). You can verify success by asking the Ampersand for its version:

If you don't have an Ampersand compiler, but docker runs on your machine, you can simply use the most recent ampersand compiler by

Use Git to create a local clone of these repositories. Git is preferred over copying the files, because you can repeatedly use it to ensure you get the right version.

5. Avoiding to install Haskell

To build an Ampersand compiler, you need Haskell. However, you can also use the latest executable Ampersand-compiler, which is published periodically on . This way, you don't need to build Ampersand and you can avoid installing Haskell.

6. Creating an Ampersand compiler

If you want, create your own Ampersand compiler. This way, you can pick a version by which to generate RAP4.

7. Installing LaTeX and GraphViz

LaTeX and GraphViz are called by RAP4 to generate documentation. Therefore, both must be installed on the server.

8. Generating the RAP4 application

You need to generate RAP4 to facilitate regular updates to the system.

9. Local Settings

Some local settings may apply, which are brought together in one file. Use this to administer things like database account(s) and PHP time limit, logging, etcetera.

10. Last minute changes before going to production

Things that are necessary for testing and development, such as logging, can be changed in the production version. These things are usually adapted shortly before going to production.