This is an old revision of the document!


Submitting the job

Before submitting the job you should run 2 commands to prepare the creation of a docker container:

umask a+r
export PATH=$PATH:/bin

The first command adds “readable for all” permission flags to the default mask of new files. Be aware of the security implications. Anyone will be able to read files you created in this session, so best use it only when submitting a Jupyter job.

The second command adds the path /bin to the list of paths where the shell looks for executables.

Below is an example of a Jupyter job script file that can be submitted using bsub.

#!/bin/sh  
#BSUB -q mpi  
#BSUB -app djupyter  
#BSUB -ISs  
  
jupyter.lsf

Usage of -app (docker application djupyter should be used) and -ISs (interactive session) are mandatory. Using the prepared script jupyter.lsfis highly recommended, since it is basically everything that is needed in most cases to run a Jupyter server and to start using it.

After a job submission, the output of the container will follow. All further instructions are within the output. The next chapters detail those instructions.

Environment

Inside the docker container you will have access to the folder where you did submit the job from. This folder will be your $HOME directory inside a container.

Connecting to the web interface

After the successful launch of the container you should see instructions like following:

[Jupyter] container is up. Preparing the environment...

[Jupyter] running jupyter notebook...

[Jupyter] jupyter is up. You can access it by following the instructions.

[Jupyter] 1. Open a new terminal and tunnel port 8888 from the host gwdc047

[Jupyter] a) With direct access to nodes: ssh -L 0.0.0.0:8888:172.18.0.2:8888 yourlogin@gwdc047.gwdg.de -N

[Jupyter] b) Without direct access: ssh -L 0.0.0.0:8888:0.0.0.0:8640 yourlogin@login.gwdg.de ssh -L 0.0.0.0:8640:0.0.0.0:8640 gwdu101 ssh -L 0.0.0.0:8640:172.18.0.2:8888 -N gwdc047

[Jupyter] 2. Open the link in a browser: http://0.0.0.0:8888/?token=ce28b2500a5f2c0e637c7c8a67fa318155e4c36bb3e5608b

The important part here is step 1. If you can directly log in to the computing nodes like gwdc047.gwdg.de, then you can run the command 1a in the terminal, otherwise run the command 1b. After that the corresponding port will be tunneled and the terminal becomes unavailable. Don’t close it! If everything is done right, then just open the link in your browser mentioned in step 2 and you should be able to access your Jupyter instance.

Installing python packages

In order to install Python packages, create a Terminal in Jupyter (New->Terminal). After that you will be redirected to a web page with a virtual shell inside the container, where you can execute commands.

You have no root access inside the container, therefore you should install Python packages in your $HOME directory (remember that the $HOME inside the container is actually the job submission folder). You can do this with the next command using pip3:

pip3 install --user package_name

After that, the package will be installed in $HOME/.local and will be available after the container is destroyed. In order to have these packages when you run other instances of Jupyter, you should submit the job from the same folder.

This website uses cookies. By using the website, you agree with storing cookies on your computer. Also you acknowledge that you have read and understand our Privacy Policy. If you do not agree leave the website.More information about cookies