Using containers on M3

What are containers?

For our purposes, a "container" can be defined as a self-contained file system that allows you to run a particular program across multiple operating systems. This allows for more reproducable software and thus reproduceable experiments, though containers can be a little awkward to use. On M3, users are welcome to play with containers for themselves (see our guide below), but the M3 admins often prepare containerised software in the form of modules that are much simpler to use. Please contact our help desk if you'd like to request a particular container to be installed.

Docker

The usage of Docker is unsupported on M3. Fortunately, Apptainer is supported on M3, almost entirely replaces Docker, and is able to work from existing Docker images. The biggest feature provided by Docker that is missing from Apptainer is Docker Compose. You are welcome to try a third-party solution called Singularity Compose for mimicking docker-compose in Apptainer, but we do not provide support for it.

Apptainer

Apptainer (formerly Singularity) simplifies the creation and execution of containers, ensuring software components are encapsulated for portability and reproducibility.

Apptainer is an alternative container solution to Docker that is more compatible with M3. The apptainer command is already installed systemwide, so you don't need to load any modules to use it. See the Apptainer documentation for a user guide.

If you want to use Apptainer on M3, note the following:

• You can use apptainer pull on M3 to e.g. pull existing Docker images from Docker Hub.
• You unfortunately can not apptainer build from a definition (.def) file on M3. Instead, you would need to build the container somewhere off of M3, then copy it over to M3.
• You can convert a Dockerfile to an Apptainer definition file using spython recipe. See https://singularityhub.github.io/singularity-cli/. You will need to install spython for yourself.
• If you are pulling on M3, do it via a Slurm job and do export APPTAINER_CACHEDIR=/tmp/apptainer-cache before running the apptainer pull command. This ensures your home directory does not run out of space due and the pull will actually succeed.