Time: July 31st 2022 until August 1st 2022 9am
Affected: Users of jupyter-cloud.gwdg.de
Impact: Service is available again
The failure of the Jupyter-Cloud service was corrected today just before 9am.
The service suffered a failure on Sunday at 8am when an automatic cleanup process removed the Docker image necessary for launching Jupyter notebook server containers for users. As a result successfully logged in users were unable to start their notebook servers and received a corresponding error message in JupyterHub. After re-downloading the image in question the service was immediately available again. The cleanup process has been disabled and will be reworked.
A cleanup process exists to remove older Notebook images to reduced storage usage. Older images exists because of updates performed by a new semi-automatic process, which has been setup last week in order to perform updates and changes to the Notebook images quicker and more often in the future. Unfortunately during this endeavor the malfunctioning cleanup process was also installed.