Batch jobs

A batch job is a job in which you submit a particular script (set of commands) to run on compute nodes with zero interactivity. I.e. once the job starts, you cannot interact with it.

To submit a batch job, you need a batch script (AKA a "job submission script"). Read our guide on editing files in M3 if you haven't already.

An example batch script is below:

#!/bin/bash
#SBATCH --job-name="Example job name!"
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=1
#SBATCH --mem=1G

echo "This job has started!"
# Sleep to pretend we're doing some useful work...
sleep 10
echo "This job has ended!"

Let's explain the structure of this script in detail.

Shebang

You must have a shebang on the first line. This tells Slurm which "interpreter" to use to run your script, i.e. what "language" your script is written in. Since we're focusing on Bash scripts, we will always have a Bash shebang:

#!/bin/bash

SBATCH directives

In Bash, a line starting with # is a comment. Comments are useful to indicate what a command is doing or why something has been done in a particular way. Bash will ignore that line entirely when executing.

However, Slurm lets you put special comments at the top of the file to indicate your requested resources for this job script. In our example, these lines were

#SBATCH --job-name="Example job name!"
#SBATCH --time=1:00:00
#SBATCH --mem=1G

These are #SBATCH "directives". You should always place all of your #SBATCH lines at the top of the file right after the shebang. Do not put actual code between #SBATCH lines.

On each line, the stuff after #SBATCH corresponds to a valid argument/flag for the sbatch command. You can find a complete list of sbatch options in the official Slurm sbatch page. Our own guide to setting these flags on M3 can be found at Specifying resources in Slurm. In the above example, we did:

• --job-name="Example job name!": give a name to the job that you can see when looking in the job queue or job history.
• --time=1:00:00: ask for this allocation for 1 hour. Your job will be killed if it takes longer than this.
• --mem=1G: ask for 1 GB of memory.

Your actual commands

Everything after the #SBATCH directives is the actual commands you wish to run. This could be a simple bash script, or it could be something more complicated that loads some software, and uses Python to run some other script.

Submitting a batch job

Once you are happy with your batch script, you can submit it with

sbatch my-script.slurm

where my-script.slurm is the name of your job script.

Viewing the batch job's output

See Monitoring your jobs.