Using Julia on the SCC

Introduction

So, you want to speed up your Julia code? In order to make use of HPC resources, the first step is to bring your workflow to the cluster. Since this can tedious to figure out, this article provides how to. Sections marked with a diamond 💎 contain additional tips and debugging help if something is not working out–of–the–box (i.e., copy–paste is failing). Note that this article is not a general guide to get started on the cluster nor is it a general guide to get started on the cluster nor is it a guide to Julia.

Find more information about this article here.

Overview

1. Installing Julia
2. Interactive Usage
3. Non–Interactive Usage

Installing Julia

In general, there are two options for installing software on the cluster: Via the module system, common software and environment information is installed for all users. If you need a specific software or Julia version not available via this method, you can also install the specific Julia version via the package management tool Spack. (Of course, you can also compile from source, the third option not covered here.)

Via Module

To see all available pre–installed Julia versions and environments, run module spider julia. To use a listed version, run module load julia/1.5.2.

Via Spack

To install Julia via Spack, have a look at the more detailed description and run

module load spack-user
source $SPACK_USER_ROOT/share/spack/setup-env.sh
spack install julia@1.8.1 # For installation.
spack load julia@1.8.1 # For loading.

Irrespective of how you install Julia, before executing Julia in a new environment (such as on a computing node), you first need load it, depending on the istallation via (via module load julia or executing the lines above).

Checking Installation and Paths💎

To check the whether everything is as you would expect, you can start the Julia REPL (typing julia) and print the following:

  • VERSION Which Julia version is running?
  • versioninfo() Which version is running, on which platform, and where do environment paths point to (e.g., LD_LIBRARY_PATH and JULIA_ROOT?
  • Base.active_project() Which Project.toml is currently activated?
  • DEPOT_PATH, LOAD_PATH

 

As usual, you can open the package manager with ] and activate another project environment, e.g. with activate .. To run compute-intensive jobs, schedule them with Slurm. Both interactive and non-interactive usage of Julia with Slurm will be covered in the following.

Using Julia REPL Interactively

To use Julia interactively, first start an interactive Slurm job and then start Julia. To start the interactive Slurm job, run
srun –pty -p medium -N 1 -c 16 /bin/bash
with the arguments
–pty to start a pseudo-terminal
-p for the partition (here medium, use -p gpu for GPU cores)
-N 1 the minimum amount of nodes
-c 16 the number of CPU cores

Checking with hostname, you see node you are on. After loading Julia, start it with julia –threads 4 –project=”PATH_TO_YOUR_PROJECT/Project.toml”

💎 “No such file or directory“ for project file

Your project file cannot be found. To ensure that you are using the correct path, log into the frontend and check the path within the project with pwd -P. This gives you the physical path (without softlinks), which should work.

Using Julia Non–Interactively

Submit your non–interactive Julia computation job via the Slurm scheduler. In the following, an example is provided with For intance, create a folder in which you load a Julia using BenchmarkTools (that contains a folder, no other package is used).

#!/bin/bash
#SBATCH –job-name=julia-hello-world
#SBATCH –nodes=1 # total number of nodes
#SBATCH –ntasks=1 # total number of tasks
#SBATCH –cpus-per-task=1 # cpu cores per task
#SBATCH –time=00:01:00 # total run time limit (HH:MM:SS)
#SBATCH –mail-type=begin # send mail when job begins
#SBATCH –mail-type=end # send mail when job ends
#SBATCH –mail-user=YOURNAME@mail.com

# Prepare the environment.
module load spack-user
source $SPACK_USER_ROOT/share/spack/setup-env.sh
spack load julia@1.8.2

# Run the script.
julia –project=”PATH_TO_YOUR_FOLDER/Project.toml” hello-world.jl

using BenchmarkTools

function myprinter(str::String)
println(str)
end

function sleepingfn()
sleep(2)
end

a = “Hello, world!”
myprinter(a)
@btime sleepingfn()

General Tips

💎 Invalid history file format

The following can happen when switching between different versions of Julia:
ERROR: Invalid history file
(/usr/users/YOUR_USERNAME/.julia/logs/repl_history.jl) format: If you have a history file left over from an older version of Julia, try renaming or deleting it.

If this bothers you, simply remove the Julia REPL history with rm
/usr/users/YOUR_USERNAME/.julia/logs/repl_history.jl.

💎 Installing new packages takes long
Opening the registry with ] and adding something (e.g., add Example) takes a long time during the updatating registries step? First, ensure that the correct Julia uses the correct package server (with
export JULIA_PKG_SERVER=eu-central.pkg.julialang.org). Furthermore, you can create a startup file ~/.julia/config/startup.jl containing

import Pkg
Pkg.UPDATED_REGISTRY_THIS_SESSION[] = true

Then, the registry will not automatically get updated. It will update if you explicitly type ] registry up.(Don’t forget to do this from time to time.) Note that you can even do fancier startup files! You can also start Julia with a specific startup file, julia -i –project=”.” –startup-file=no “startup.jl” (use i for interactive mode).

Acknowledgements

Thanks to Carsten Bauer for teaching so much about Julia!

Author

Dorothea Sommer

Categories

Archives

--