Working on multiple open-source projects written in multiple programming languages, frameworks, and environments, I often face the requirement to install additional packages to test changes I might be doing to my local repository checkout. Since I try to minimize the amount of software installed on my workstation, I would have to start a virtual machine (VM), install the required bits there, and solve the mounting of my working tree into the VM.

Lately, I use containers for that task and the process got easier and faster. Let's look at how test execution can be isolated into container-based environment. If you are not interested in the thought process, just find the final Dockerfile at the end.

For the Django project, the guidance for running tests includes

$ cd django-repo/tests
$ PYTHONPATH=..:$PYTHONPATH ./runtests.py

Executing these commands on my system leads to

Please install test dependencies first: 
$ pip install -r requirements/py2.txt

and some people including me are not inclined to do that. But we'll be happy to meet those requirements in separate chroot, and containers are really just chroots with additional features.

We want to minimize the access we give to the processes executing the tests, so we will run the processes as user nobody:

$ docker run -u nobody ...

For that to work, all the parent directories of the Django checkout directory need to be at least accessible by other since nobody usually is not member of any useful groups — we look for drwx--x--x in ls -l output.

However, attempt to run a quick check fails:

$ docker run -u nobody -v $PWD:/django:ro -ti fedora cat /django/tests/requirements/py2.txt
cat: /django/tests/requirements/py2.txt: Permission denied

My host is Fedora and my docker daemon is SELinux enabled — there is

OPTIONS='--selinux-enabled --icc=false'

in /etc/sysconfig/docker. By default each container gets SELinux type svirt_lxc_net_t and unique MLS values. The ps axZ might show for example

system_u:system_r:svirt_lxc_net_t:s0:c119,c856

for processes running in container, where the c119,c856 part would be different for every container to isolate them frome ach other. And type svirt_lxc_net_t is not allowed to access content in user home directories, SELinux type user_home_t.

One solution is to change the SELinux label of the project git working tree to a context that containerized applications would be able to access, for example tmp_t. Note that plain

$ chcon -R -t tmp_t .

will change the type but subsequent restorecon will likely change it back to user_home_t. To prevent that from happening, either use semanage fcontext ... to specify this new type for path where the checkout is located, or store the working trees in separate location outside of home directory where they won't be relabelled.

Another possibility is to use spc_t SELinux type for the container processes with

$ docker run --security-opt=label:disable ...

That type is similar to unconfined_t by having wide set of operations allowed.

Our containerized processes are now able to access the Django tests/ directory but we still need to prepare the container environment a bit because the default Fedora image does not even contain python:

$ docker run -u nobody --security-opt=label:disable -v $PWD:/django:ro --rm -ti \
    fedora bash -c 'cd /django/tests && PYTHONPATH=..:$PYTHONPATH ./runtests.py'
/usr/bin/env: python: No such file or directory

It's time to create container image which will have python installed. With tests/Dockerfile

FROM fedora
RUN dnf install -y python && dnf clean all

we can build image django-tests with

$ docker build -t django-tests tests
[ ... see python installed ... ]
Successfully built 9273aa7438bc

Now we can repeat our ./runtests.py attempt, just using the new image with python instead of the vanilla Fedora base image:

$ docker run -u nobody --security-opt=label:disable -v $PWD:/django:ro --rm -ti \
    fedora django-tests -c 'cd /django/tests && PYTHONPATH=..:$PYTHONPATH ./runtests.py'
Please install test dependencies first: 
$ pip install -r requirements/py2.txt

We are at the same point we were at the beginning on the host, except we will happily install the needed software in the container.

We could certainly run the pip install command before running ./runtests.py in run-time but that would mean installing the packages over and over again upon every test run. It usually takes me more than one test execution to get my patch right, so making that part faster is a reasonable goal. We can run the pip install in build-time and make the installed packages part of the container image:

FROM fedora
RUN dnf install -y python python-devel python-cffi gcc redhat-rpm-config && dnf clean all
ADD requirements/* /django-requirements/
RUN pip install -r /django-requirements/py2.txt

We take advantage of the fact that during docker build, the tests/ subdirectory is available as build context, so, we can copy the tests/requirements/ files to the image and run pip install based on them.

However, the above is likely to fail since to build the python modules from sources, -devel rpm packages for the C libraries are often needed. Let's try to install as many packages as possible as rpms, and only then fallback to installing with pip:

RUN mkdir /django-requirements && echo -e '#!/bin/bash          \n \
        r () {                                                  \n \
                echo "# Reading $1" >&2                         \n \
                while read i ; do                               \n \
                        s="${i#-r }"                            \n \
                        if [ "$i" != "$s" ] ; then              \n \
                                r "$s"                          \n \
                                echo "# Back to $1" >&2         \n \
                        elif [ "$i" == "${i/##/}" ] ; then      \n \
                                echo "python-$i"                \n \
                                echo "python-${i,,}"            \n \
                        fi                                      \n \
                done < "$1"                                     \n \
        }                                                       \n \
        readarray -t p < <( r "$1")                             \n \                                                    
        dnf install -y --setopt=strict=0 python-devel gcc redhat-rpm-config "${p[@]}"'  \
                > /django-requirements/dnf-install-python-requirements
ADD requirements/* /django-requirements/
RUN ( cd /django-requirements && bash ./dnf-install-python-requirements py2.txt )
RUN pip install -r /django-requirements/py2.txt

We also have to install some additional packages (python-devel, gcc) needed to compile from sources.

To make the test execution easier, let's make a simple script which will set the PYTHONPATH accordingly:

RUN ( echo '#!/bin/bash' ; echo 'cd /django/tests && PYTHONPATH=.. ./runtests.py "$@"' ) > /bin/django-runtests
RUN chmod a+x /bin/django-runtests

We also might want to stop python from attempting to write .pyc files with

ENV PYTHONDONTWRITEBYTECODE 1

which is bound to fail anyway, especially when we force the execution as nobody:

USER nobody

The whole tests/Dockerfile will thus be

FROM fedora
RUN mkdir /django-requirements && echo -e '#!/bin/bash		\n \
	r () {							\n \
		echo "# Reading $1" >&2				\n \
		while read i ; do				\n \
			s="${i#-r }"				\n \
			if [ "$i" != "$s" ] ; then		\n \
				r "$s"				\n \
				echo "# Back to $1" >&2		\n \
			elif [ "$i" == "${i/##/}" ] ; then	\n \
				echo "python-$i"		\n \
				echo "python-${i,,}"		\n \
			fi					\n \
		done < "$1"					\n \
	}							\n \
	readarray -t p < <( r "$1")				\n \
	EXTRA="python-devel gcc redhat-rpm-config"		\n \
	dnf install -y --setopt=strict=0 $EXTRA "${p[@]}" && dnf clean all'	\
		> /django-requirements/dnf-install-python-requirements
ADD requirements/* /django-requirements/
RUN ( cd /django-requirements && bash ./dnf-install-python-requirements py2.txt )
RUN pip install -r /django-requirements/py2.txt
RUN ( echo '#!/bin/bash' ; echo 'cd /django/tests && PYTHONPATH=.. ./runtests.py "$@"' ) > /bin/django-runtests
RUN chmod a+x /bin/django-runtests
ENV PYTHONDONTWRITEBYTECODE 1
USER nobody

We can build the final image with

$ docker build -t django-tests tests

and run the tests for example using

$ docker run --rm -ti --security-opt=label:disable -v $PWD:/django:ro django-tests /bin/django-runtests -v2 admin_views
Testing against Django installed in '/django/django' with up to 4 processes
Importing application admin_views
Creating test database for alias 'default' (':memory:')...
[ ... ]
Ran 296 tests in 19.500s
OK (skipped=22)
[ ... ]

The steps above were working on Fedora 23 with docker-1.10.3-16.gita41254f.fc23.x86_64 and Django 1.9.x and master (before 1.10) branches.

Using containers to run tests can not just be faster than using virtual machines and keep workstation tidy from dependencies of various projects, it can also be used to easily test the project on different operating systems (OS), with different versions of libraries, taking into account OS packages (rpms, in case of Fedora, CentOS, or Red Hat Enterprise Linux) that might be installed on machines where the project will be deployed.