Testing Linux Packages with Docker

jonaka-docker

During my apprenticeship at ownCloud  i had the opportunity to write a proof of concept for installation testing of ownCloud Linux packages for various distributions with the help of docker.

Requirements:

The goal was to create an testframework where each test could be ran individually and also locally on the developers machine. During my apprenticeship i used docker a lot and learned it is an awesome software  to do things quickly without the hassle of booting up VM’s. To get an ownCloud instance running all i needed was:

docker compose up

with the ownCloud Image from: https://github.com/owncloud-docker/server

So naturally  i went with docker for the testing framework i wanted to create.

Getting dirty:

I wanted to test the installation of ownCloud packages on four different Distros: ubuntu:16.04, CentOS7, Debian9 and openSuse Leap 42.3 seemed feasible.

You can find the projects Source Code right here:

https://github.com/Kawohl/abschlussprojekt

The general idea  was to fire up an docker container for each Distro and run the installation for the ownCloud Linux Packages in each container individually.

I wrote Scripts to do ownCloud Package installations for each distro. These Scripts are mounted into the corresponding containers. After the installation, the Scripts write an output to a log file on the host.

All in all, I had only one 35 hours to complete the scripts and write an elaborate documentation for school.  Unfortunately, i had to write in German because of school regulations.  Abschlussprojekt

[action-button color=”blue” title=”Download the Documentation here:” subtitle=”PDF” url=”https://blog.jonaka.de/wp-content/uploads/2018/05/Abschlussprojekt_w2fjke.pdf”]

Why do installation Tests?

The ownCloud Server is a PHP application running typically on Linux with an Apache web server and a MYSQL/Mariadb database. In some cases, administrators prefer Postgres as a Database and for testing purposes with a minimal amount of Data they might want to use sqlite as a backend.  Different distros offer by default different versions of the Software dependencies, meaning different php Versions, different versions of databases, and even the default apache configuration may differ from distro to distro. For everything to work nicely together ownCloud needs a variety of additional php extensions :

PHP Extensions

NameDescription
CtypeFor character type checking
cURLUsed for aspects of HTTP user authentication
DOMFor operating on XML documents through the DOM API
GDFor creating and manipulating image files in a variety of different image formats, including GIF, PNG, JPEG, WBMP, and XPM.
iconvFor working with the iconv character set conversion facility.
intlIncreases language translation performance and fixes sorting of non-ASCII characters
JSONFor working with the JSON data-interchange format.
libxmlThis is required for the _DOM_, _libxml_, _SimpleXML_, and _XMLWriter_ extensions to work. It requires that libxml2, version 2.7.0 or higher, is installed.
Multibyte StringFor working with multibyte character encoding schemes.
OpenSSLFor symmetric and asymmetric encryption and decryption, PBKDF2, PKCS7, PKCS12, X509 and other crypto operations.
PDOThis is required for the pdo_msql function to work.
PharFor working with PHP Archives (.phar files).
POSIXFor working with UNIX POSIX functionality.
SimpleXMLFor working with XML files as objects.
XMLWriterFor generating streams or files of XML data.
ZipFor reading and writing ZIP compressed archives and the files inside them.
ZlibFor reading and writing gzip (.gz) compressed files.

And for the database integration:

Database Extensions

NameDescription
pdo_mysqlFor working with MySQL & MariaDB.
pgsqlFor working with PostgreSQL. It requires PostgreSQL 9.0 or above.
sqliteFor working with SQLite. It requires SQLite 3 or above. This is, usually, not recommended, for performance reasons.

 

That said: Whats the Problem?

Usually, sysadmins would just install the packages provided by the distro for all dependencies needed to get ownCloud running.

Lets have a closer look:

To get the ownCloud Server running on an ubuntu  16.04 we would need following packages:

Package Installation, PHP, and Apache configuration.

apt install -y apache2 mariadb-server libapache2-mod-php7.0 php7.0-gd php7.0-json php7.0-mysql php7.0-curl php7.0-intl php7.0-mcrypt php-imagick php7.0-zip php7.0-xml php7.0-mbstring

Then we start the Apache-Webserver:

service apache2 start

ownCloud installation and Apache Configuration

Then we get the ownCloud package:

    • Get the Release key of the ownCloud Release:
wget -nv https://download.owncloud.org/download/repositories/production/Ubuntu_16.04/Release.key -O Release.key< apt-key add - < Release.key
    • Add ownCloud to the repository list
echo 'deb http://download.owncloud.org/download/repositories/production/Ubuntu_16.04/ /' > /etc/apt/sources.list.d/owncloud.list
    • Update the repository list:
apt update
    • install the ownCloud package:
apt install -y owncloud-files
    • add the ownCloud vhost config to the apache webserver. So we write to the file:
      /etc/apache2/sites-available/owncloud.conf

<Directory /var/www/owncloud/>
Options +FollowSymlinks
AllowOverride All

<IfModule mod_dav.c>
Dav off
</IfModule>

SetEnv HOME /var/www/owncloud
SetEnv HTTP_HOME /var/www/owncloud

</Directory>

    • Create a Symbolic Link to enable the config:
ln -s /etc/apache2/sites-available/owncloud.conf /etc/apache2/sites-enabled/owncloud.conf
    • Enable phpmodules
a2enmod rewrite
    • Restart apache
service apache2 start

Database Configuration:

Create Database:

mysql -u root -e "create database owncloud"

Create ownCloud user:

mysql -u root -e "create user 'ownclouduser'@localhost identified by 'admin'"

grant the ownCloud user rights

mysql -u root -e "GRANT ALL PRIVILEGES ON owncloud. * TO 'ownclouduser'@'localhost'"

ownCloud Installation:

And finally we use the ownCloud occ command to finalize the installation

sudo -u www-data php /var/www/owncloud/occ maintenance:install --database "mysql" --database-name "owncloud" --database-user "ownclouduser" --database-pass "admin" --admin-user "admin" --admin-pass "admin"

Lets check if we succeeded:

First we need to install some additional packages:

apt install jq lynx

lynx is a textbased web browser that allows us to browse the ownCloud installation for the status.php within the docker container.  When ownCloud is correctly installed the status.php should show something like this:

{"installed":true,"maintenance":false,"needsDbUpgrade":false,"version":"10.0.6.1","versionstring":"10.0.6","edition":"Community","productname":"ownCloud"}

And then we run this command to check if ownCloud was installed successfully:

(echo "SUCCESS:$(lynx --dump localhost/owncloud/status.php| jq -r .versionstring ) installed! System $PRETTY_NAME" || echo "FAIL: Installation failed! System $PRETTY_NAME") >> /logs/server.install.log 2>&1

 

This results with a line in the server.install.log file : 

SUCCESS:10.0.4 installed! System Ubuntu 16.04.3 LTS

Summary package installation in ubuntu 16.04

The ownCloud package installation is pretty much forward:

Please Note: This is only for testing purposes and in no way an recommendation for deployments.

Docker:

To run the single installation process in a container we do:

docker run -ti --rm=true -v $(pwd)/server/ubuntu16.04/latestproduction/install.sh:/install.sh -v $(pwd)/logs/:/logs ubuntu:16.04 sh install.sh

In the root folder of the project

This starts the docker container and removes it when the script has run to save disk space. It mounts the install script for the latest ownCloud server for the ubuntu packages, runs it and outputs the results of the test to the server.install.log located in the root of the project.

The Folder structure looks like this in my Texteditor (Sublime)

jonaka.de-scripstructure

One Script to rule them all

In the root folder, there is a run.sh file that starts every container sequentially and starts the package installation for each and any distro.

Some remarks….

During writing these tests I ran into a couple of minor issues. Centos7 comes with php 5.4 by default. PHP 5.4 isn’t supported since 3rd of September 2015!

You can either compile the needed php or download it from some external repos:  Check out, for instance, Remi’s Repos:

https://blog.remirepo.net/post/2016/02/14/Install-PHP-7-on-CentOS-RHEL-Fedora

Dear openSuse!

I love your Distro and think you do awesome work. And I get it: different Distros have different package names: I can live with that: But:

apache2-mod_php7

Come on! Every other package name with php in it is with a hyphen and all of the sudden you add an underscore? That seems weird and adds IMHO unnecessary complexity.  I understand that this package is Apache related, but also a

zypper se php7

doesn’t give you the package name.  Searching for the package is an unnecessary p.i.t.a.. and has cost me some time.

tl;dr: I wrote a small little tool to test if packages install properly on different Linux Distros.