The MPI (Message Parsing Interface) communication protocol allows applications to run on multiple cores or nodes simultaneously (so-called dispersed calculations) if it is compatible with the protocol. However, the MPI can also be used to enable multiple instances of one program within a single task.
Separation of the task into multiple nodes may be useful if one node does not have the expected number of free cores, or if the calculation speed is limited by the RAM capacity.
Options to run jobs on multiple cores/nodes:
-c <cpus>
--cpus-per-task=<cpus> Number of cpus assigned to each job
-n <ntasks>
--ntasks=<ntasks> Number of thraeds to run
-N <nodes>
--nodes=<nodes> Number of nodes
For MPI tasks, it is very important to distinguish between the paramertes "-c" and "-n".
Giving "-c 10" without "-n" will run 10 independent MPI processes - as if 10 different tasks were run, each on a single core.
Giving "-c 1 -n 10" of the MPI processes will run the proper start-up of 10 MPI processes.
There are two MPI implementations available:
An example of an MPI script running 10 processes on two nodes:
#!/bin/bash
#SBATCH -N2
#SBATCH -n10
#SBATCH --ntasks-per-node=5
#SBATCH -c1
#SBATCH --mem-per-cpu=300
#SBATCH -t10
module load Python
module load OpenMPI
mpiexec python3 symulacja.py
The recommended method running intel MPI is using srun after prior indication of the location of the slurm PMI library.
An example script for a task run with Intel MPI:
#!/bin/bash
#SBATCH -N2
#SBATCH -n10
#SBATCH --mem=10gb
#SBATCH --time=1:00:00
#SBATCH --job-name=intel_MPI_test
export I_MPI_PMI_LIBRARY=/opt/slurm/current/lib64/libpmi.so
module load intel/2021a
srun intel_MPI_script.sh