Differences

This shows you the differences between two versions of the page.

--- en:services:application_services:high_performance_computing:ipython_parallel [2019/02/18 16:49]
akhuziy
+++ en:services:application_services:high_performance_computing:ipython_parallel [2019/07/22 14:54] (current)
akhuziy
@@ Line 9: / Line 9: @@
 <code>
-bsub -ISs -q int singularity shell -B /opt sjupyter.simg
+srun --pty -p int singularity shell -B /opt sjupyter.sif
 </code>
-To be able to run LSF commands within the container, the LSF variables need to be initialized:
+To be able to run Slurm commands within the container, additional libraries and directories should be bound into the container:
 <code>
-. /opt/lsf/conf/profile.lsf
+singularity -B /var/run/munge,/run/munge,/usr/lib64/libmunge.so.2,/usr/lib64/libmunge.so.2.0.0,/etc/profile.d/slurm.sh ...
+</code>
+Also you need to add //slurmadmin// user to the container when building the image, with following commands:
+<code>
+echo "slurmadmin:x:300:300::/opt/slurm/slurm:/bin/false" >> /etc/passwd
+echo "slurmadmin:x:300:" >> /etc/group
 </code>
@@ Line 23: / Line 29: @@
 <code>
-ipython profile create --parallel --profile=lsf
+ipython profile create --parallel --profile=myslurm
 </code>
-This will create the profile at ''%%$HOME/.ipython/profile_lsf%%''. Now you need to configure it for LSF.
+This will create the profile at ''%%$HOME/.ipython/profile_myslurm%%''. Now you need to configure it for Slurm.
-Add following config lines to the file ''%%$HOME/.ipython/profile_lsf/ipcluster_config.py%%'':
+Add following config lines to the file ''%%$HOME/.ipython/profile_myslurm/ipcluster_config.py%%'':
 <code>
-c.IPClusterEngines.engine_launcher_class = 'LSFEngineSetLauncher'
+c.IPClusterEngines.engine_launcher_class = 'SlurmEngineSetLauncher'
-c.IPClusterStart.controller_launcher_class = 'LSFControllerLauncher'
+c.IPClusterStart.controller_launcher_class = 'SlurmControllerLauncher'
-c.LSFControllerLauncher.batch_template_file ='lsf.controller.template'
+c.SlurmControllerLauncher.batch_template_file ='slurm.controller.template'
-c.LSFEngineSetLauncher.batch_template_file = 'lsf.engine.template'
+c.SlurmEngineSetLauncher.batch_template_file = 'slurm.engine.template'
 </code>
 and comment out the following parameters:
 <code>
-#c.LSFControllerLauncher.batch_template = "..."
+#c.SlurmControllerLauncher.batch_template = "..."
-#c.LSFEngineSetLauncher.batch_template = "..."
+#c.SlurmEngineSetLauncher.batch_template = "..."
 </code>
-Add the following line to ''%%$HOME/.ipython/profile_lsf/ipcontroller_config.py%%'':
+Add the following line to ''%%$HOME/.ipython/profile_myslurm/ipcontroller_config.py%%'':
 <code>
@@ Line 48: / Line 54: @@
 </code>
-IPython Parallel is almost ready to use. For submitting LSF jobs in a specific queue and with additional parameters, create templates for batch jobs in the directory you want to start the container using the names specified in the configuration file, i.e. ''%%lsf.controller.template%%'' and ''%%lsf.engine.template%%''.
+IPython Parallel is almost ready to use. For submitting Slurm jobs in a specific queue and with additional parameters, create templates for batch jobs in the directory you want to start the container using the names specified in the configuration file, i.e. ''%%slurm.controller.template%%'' and ''%%slurm.engine.template%%''.
-lsf.controller.template:
+slurm.controller.template:
 <code>
 #!/bin/bash
-#BSUB -J ipcontroller
+#SBATCH -p medium
-#BSUB -o ipcontroller.%J
+#SBATCH -J ipcontroller
-#BSUB -n 1
+#SBATCH -o jupyterhub-gwdg/current.ipcontroller.log
-#BSUB -W 5:00
+#SBATCH -n 1
-#BSUB -q int
+#SBATCH -t 1:00:00
-export PATH=$PATH:/usr/bin
+export PATH=$PATH:/usr/bin:/usr/local/bin
-export PATH=$PATH:/cm/shared/apps/singularity/3.0.2/bin/
+export PATH=$PATH:/cm/shared/apps/singularity/3.2.0/bin/
-singularity exec sjupyter.simg ipcontroller --profile-dir={profile_dir} --location=$HOSTNAME
+singularity exec sjupyter.sif ipcontroller --profile-dir={profile_dir} --location=$HOSTNAME
 </code>
-lsf.engine.template:
+slurm.engine.template:
 <code>
 #!/bin/bash
-#BSUB -n {n}
+#SBATCH -p medium
-#BSUB -o ipengine.%J
+#SBATCH -J ipengine
-#BSUB -W 5:00
+#SBATCH -n {n}
-#BSUB -q mpi
+#SBATCH -o jupyterhub-gwdg/current.ipengine.log
-#BSUB -R span[ptile='!']
+#SBATCH -t 1:00:00
-#BSUB -R same[model]
-#BSUB -a intelmpi
-export PATH=$PATH:/usr/bin
+export PATH=$PATH:/usr/bin:/usr/local/bin
-export PATH=$PATH:/cm/shared/apps/singularity/3.0.2/bin/
+export PATH=$PATH:/cm/shared/apps/singularity/3.2.0/bin/
-export I_MPI_ROOT=/cm/shared/apps/intel/compilers_and_libraries/2017.2.174/mpi
-export PATH=$PATH:$I_MPI_ROOT/intel64/bin
-export LD_LIBRARY_PATH=$I_MPI_ROOT/intel64/lib
-mpirun.lsf singularity exec sjupyter.simg ipengine --profile-dir={profile_dir}
+srun singularity exec sjupyter.sif ipengine --profile-dir={profile_dir}
 </code>
@@ Line 103: / Line 104: @@
 For ''%%<host>%%'' insert the node where the container is running. Open the link in the Jupyter output in your browswer.
-To start the cluster, use the IPython Clusters tab in the Jupyter interface, select the lsf profile and amount of processes and click **start**. You will be able to see the engines running with the bsub command.
+To start the cluster, use the IPython Clusters tab in the Jupyter interface, select the myslurm profile and amount of processes and click **start**. You will be able to see the engines running with the "squeue -u $USER" command.
 To test if it is working, simply run following script in the Jupyter notebook:
@@ Line 109: / Line 110: @@
 <code>
 import ipyparallel as ipp
-c = ipp.Client(profile="lsf")
+c = ipp.Client(profile="myslurm")
 c.ids
 c[:].apply_sync(lambda : "Hello, World")
 </code>