Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
en:services:application_services:jupyter:start [2019/05/10 14:26]
bwegman1 [What is Jupyter / Jupyter-Hub?]
en:services:application_services:jupyter:start [2019/10/08 14:22] (current)
bwegman1
Line 9: Line 9:
 Jupyter-Hub is the portal users log in to and start and manage their notebooks and associated files. Jupyter-Hub is the portal users log in to and start and manage their notebooks and associated files.
  
-Please note that you can also use [[https://​info.gwdg.de/​docs/​doku.php?​id=en:​services:​application_services:​high_performance_computing:​jupyter|Jupyter on our SCC]] if you do need direct access to data on the SCC and have medium to high compute resource requirements.+Please note that you can also use [[https://​info.gwdg.de/​docs/​doku.php?​id=en:​services:​application_services:​jupyter:hpc|Jupyter on our SCC]] if you do need direct access to data on the SCC and have medium to high compute resource requirements.
  
 ===== How to use Jupyter / Jupyter-Hub?​ ===== ===== How to use Jupyter / Jupyter-Hub?​ =====
Line 40: Line 40:
 ==== Installing additional python modules ==== ==== Installing additional python modules ====
 Additional Python modules can be installed via the terminal and the Python package manager "​pip"​. To do this, a terminal must be opened via the menu "​New"​ -> "​Terminal"​. Afterwards <​code>​pip install --user <​module></​code>​ installs a new module in the home directory. Additional Python modules can be installed via the terminal and the Python package manager "​pip"​. To do this, a terminal must be opened via the menu "​New"​ -> "​Terminal"​. Afterwards <​code>​pip install --user <​module></​code>​ installs a new module in the home directory.
 +
 +=== Installing large python modules and disk space ===
 +
 +The installation of large Python modules like "​tensorflow"​ may fail with a message "No space left on device"​. This is caused by the temporary space under "/​tmp"​ being too small for pip to work the downloaded packages. The following steps use a temporary directory in the much larger user home directory for this one installation:​
 +
 +<code bash>
 +mkdir -v ~/​.user-temp
 +TEMP=~/​.user-temp pip install --user <​module>​
 +</​code>​
 +
 +Prefixing the installation with the TEMP variable makes pip use that location for this one installation.
 +
 +=== Notebook fails to start after package installation or update, error 500 ===
 +
 +If every notebook fails to start after a package installation or upgrade (error 500) the issue can be resolved by renaming the folder ''​.local'':​
 +<code bash>
 +mv -v .local/ .local.gwdg-disable
 +</​code>​
 +This deactivates all additionally installed packages. The notebook should start again normally.
 +
 +==== Installation of additional packages and environments via Conda ====
 +
 +Management of software packages and environments with Conda requires a terminal session started from the notebook server. The terminal ist available after login via ''​New -> Terminal''​.
 +
 +Before working with ''​conda''​ commands the necessary conda functions need to be loaded (<​b>​mind the dot at the beginning!</​b>​):​
 +<code bash>
 +. /​opt/​conda/​etc/​profile.d/​conda.sh
 +</​code>​
 +
 +=== Creating a new environment ===
 +
 +The following describes the creation of a new, simple environment ''​wikidoku'',​ the installation of the package ''​jinja2''​ as an example and how to make the environment available in the kernel selection of the notebook.
 +
 +Creating and activating the environment:​
 +<code bash>
 +conda create -y --prefix ./wikidoku
 +conda activate ./wikidoku
 +</​code>​
 +
 +As an example the package ''​jinja2''​ will be installed next. This is the step to install the desired software packages from various Conda channels.
 +<code bash>
 +conda install -y jinja2
 +</​code>​
 +
 +Next the new environment will be registered with the notebook. Terms for ''​--name''​ and ''​--display-name''​ (optional) can be chosen freely, the later being shown in the kernel selection of the notebook. The second command lists all known environments. ​ The last command exists the environment.
 +<code bash>
 +python3 -m ipykernel install --user --name wikidoku --display-name "​Python (wikidoku)"​
 +jupyter kernelspec list
 +conda deactivate
 +</​code>​
 +
 +If installation of the kernel fails with the message ''/​usr/​bin/​python:​ No module named ipykernel''​ the additional package ''​jupyter''​ needs to be installed and installation of the kernel repeated:
 +<code bash>
 +python3 -m pip install jupyter
 +</​code>​
 +
 +=== Selecting the new environment ===
 +
 +== Restarting of the notebook server ==
 +
 +After installation of a new environment it is recommended to restart the notebook server. Leave all existing terminals and close all open notebooks. In the Jupyter overview page click on ''​Control Panel''​ and stop the server by clicking ''​Stop My Server''​ and restart it with ''​Start My Server''​.
 +
 +Via ''​New ->''​ a new notebook with the new environment can be started. In existing notebooks the kernel can be changed after starting the notebook and selecting ''​Kernel -> Change Kernel''​.
 +
 +=== Installing additional kernels in an Conda environment ===
 +
 +Installing a new, independent Python kernel für the current environment is possible. As an example an older Python 2.7 kernel will be installed next.
 +
 +A new environment needs to be created and activated as per the steps above.
 +Next follows the installation of the kernel, the ''​jupyter''​ module for that kernel and finally the kernel is made available for selection in the kernel list.
 +<code bash>
 +conda install -y python=2.7
 +python3 -m pip install jupyter
 +python3 -m ipykernel install --user --name oldpython --display-name "​Python 2.7 (oldpython)"​
 +</​code>​
 +
 +The new kernel is now available for new and existing notebooks after restarting the notebook server.
 +The current kernel version can be queried from within Python:
 +<code python>
 +import sys
 +print (sys.version)
 +</​code>​
 +
 +=== Removing an environment ===
 +
 +In order to remove an environment it has to be de-registered from the notebook server and then its files removed (optional but recommended). We list the installed kernels, de-register and remove the environment'​s files:
 +
 +<code bash>
 +jupyter kernelspec list
 +jupyter kernelspec remove wikidoku
 +rm -rf ./wikidoku
 +</​code>​
  
 ==== Installing additional R packages==== ==== Installing additional R packages====