Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
en:services:application_services:high_performance_computing:running_jobs_slurm [2019/10/30 10:53]
hsommer Add missing link
en:services:application_services:high_performance_computing:running_jobs_slurm [2020/01/10 15:49] (current)
mboden [sbatch options]
Line 93: Line 93:
  
 ====  "​sbatch"​ options ​ ==== ====  "​sbatch"​ options ​ ====
 +
 +**<​nowiki>​-A all</​nowiki>​**\\
 +Specifies the account '​all'​ for the job. This option is //​mandatory//​ for users who have access to special hardware and want to use the general partitions.
  
 **<​nowiki>​-p <​partition></​nowiki>​**\\ **<​nowiki>​-p <​partition></​nowiki>​**\\
Line 106: Line 109:
 **<​nowiki>​-o <​file></​nowiki>​**\\ **<​nowiki>​-o <​file></​nowiki>​**\\
 Store the job output in "​file"​ (otherwise written to slurm-<​jobid>​). ''​%J''​ in the filename stands for the jobid.\\ Store the job output in "​file"​ (otherwise written to slurm-<​jobid>​). ''​%J''​ in the filename stands for the jobid.\\
 +
 +**<​nowiki>​--noinfo</​nowiki>​**\\
 +Some metainformation about your job will be added to your output file. If you do not want that, you can suppress it with this flag.\\
 +
 +**<​nowiki>​--mail-type=[ALL|BEGIN|END]</​nowiki>​\\
 +<​nowiki>​--mail-user=your@mail.com</​nowiki>​** \\
 +Receive mails when the jobs start, end or both. There are even more options, refer to the sbatch man-page for more information about mail types. If you have a GWDG-mail-address,​ you do not need to specify the mail-user.\\
  
 ====  Resource Selection ​ ==== ====  Resource Selection ​ ====
Line 132: Line 142:
  
 **<​nowiki>​--mem-per-cpu=<​size[units]></​nowiki>​**\\ **<​nowiki>​--mem-per-cpu=<​size[units]></​nowiki>​**\\
-Required memory per task instead of node. <​nowiki>​--mem</​nowiki>​ and <​nowiki>​--mem-per-cpu</​nowiki> ​is mutually exclusive\\+Required memory per task instead of node. <​nowiki>​--mem</​nowiki>​ and <​nowiki>​--mem-per-cpu</​nowiki> ​are mutually exclusive.\\
 === Example === === Example ===
  
Line 254: Line 264:
 ==== GPU selection ==== ==== GPU selection ====
  
-In order to use a GPU you should submit your job to the ''​gpu''​ partition, and request GPU count and optionally the model. CPUs of the nodes in gpu partition are evenly distributed for every GPU. So if you are requesting a single GPU on the node with 20 cores adn 4 GPUs, you can get up to 5 cores reserved exclusively for you, the same is with memory. So for example, if you want 2 GPUs of model Nvidia GeForce GTX 1080 with 10 CPUs, you can submit a job script with the following flags: +In order to use a GPU you should submit your job to the ''​gpu''​ partition, and request GPU count and optionally the model. CPUs of the nodes in gpu partition are evenly distributed for every GPU. So if you are requesting a single GPU on the node with 20 cores and 4 GPUs, you can get up to 5 cores reserved exclusively for you, the same is with memory. So for example, if you want 2 GPUs of model Nvidia GeForce GTX 1080 with 10 CPUs, you can submit a job script with the following flags: 
-<​code> ​+<​code>​
 #SBATCH -p gpu #SBATCH -p gpu
 #SBATCH -n 10 #SBATCH -n 10
-#SBATCH --gres=gpu:gtx1080:2+#SBATCH -gtx1080:2
 </​code>​ </​code>​
  
-You can also omit the model selection, here is an example of selecting 1 GPU of any available model: + 
-<​code>​+You can also omit the model selection, here is an example of selecting 1 GPU of any available model:<​code>​
 #SBATCH -p gpu #SBATCH -p gpu
 #SBATCH -n 10 #SBATCH -n 10
-#SBATCH --gres=gpu:1+#SBATCH -1
 </​code>​ </​code>​
 +
 +There are different options to select the number of GPUs, such as ''​%%--gpus-per-node%%'',​ ''​%%--gpus-per-task%%''​ and more. See the [[https://​slurm.schedmd.com/​sbatch.html|sbatch man page]] for details.
  
 Currently we have several generations of NVidia GPUs in the cluster, namely: Currently we have several generations of NVidia GPUs in the cluster, namely:
Line 279: Line 291:
 <​code>​ <​code>​
 #SBATCH -p gpu #SBATCH -p gpu
-#SBATCH --gres=gpu:k40:2+#SBATCH -k40:2
 </​code>​ </​code>​
 Our Tesla K40 are of the Kepler generation. Our Tesla K40 are of the Kepler generation.
Line 285: Line 297:
 <​code>​ sinfo -p gpu --format=%N,​%G </​code>​ shows a list of host with GPUs, as well as their type and count. <​code>​ sinfo -p gpu --format=%N,​%G </​code>​ shows a list of host with GPUs, as well as their type and count.
  
-=====  ​Miscallaneous ​Slurm Commands ​ =====+=====  ​Miscellaneous ​Slurm Commands ​ =====
  
 While ''​sbatch''​ is arguably the most important Slurm command, you may also find the following commands useful: While ''​sbatch''​ is arguably the most important Slurm command, you may also find the following commands useful: