Skip to content

JupyterHub at CS

JupyterHub is a multi-user JupyterLab instance.

At CS, we use these instances to run Python notebooks (mainly) for class and research. This can also be used as access to Code Server or a terminal in the browser.

The idea here is that you can access resources like GPUs, RAM and CPUs with pre-configured Docker images.

Common features

There are some similarities between the Learner and Researcher instances.

Notebook output and connection

The notebooks output cannot be saved if the connection is broken, so if you want to run a long job or you have an intermittent connection, you should consider using the command line, and not running that part of the code in a notebook.

TTYD is a good option.

Profiles

There are a number of profiles, and their availability will be in flux. More may be added too.

Custom images can be created for courses and users alike, though custom images for users will only be provided to researcher users on request.

The docker images used for these profiles are based on Jupyter Stacks.

If you find there are Jupyter extensions, packages and so on that you'd like in the profiles, or whole new profiles, please let me know.

ttyd

ttyd is included in most of the images.

This is a basic terminal interface, running on the same container as your JupyterLab instance, but accessible via the browser.

ttyd in configured to resume (or spawn) a tmux session on connection, so it is a good way to persist a terminal session.

Quick Links

Code Server

Code Server is included in most of the images, similar to VSCode, which runs in the browser.

This Code Server instance has many capabilities, here are a few:

  1. You can run multiple "workspaces", just use the file menu to open a new window.
  2. When using Chrome you can install code server as a PWA. This make it behave more like a native app.
  3. You can install extensions. The GitLab workflow extension is a good one, though make sure to use the README.md to setup your token credentials.
  4. You can run web apps on other ports, then use Code Servers built in reverse proxy to view them.

Quick Links

Home Storage

Each Jhub user is allocated home directory storage. This storage is networked, duplicated and so should be considered persistent.

However, always backup your code and work to GitLab or by using rsync to your local machine. You can also just download the files for safe keeping.

This storage will also persist between your JupyterHub sessions, but NOT between the different instances or named servers.

Logging in

  1. Click the link to the instance you want to login to.
  2. Login with your University credentials at the Azure login (if prompted).
  3. Then, follow login details for the instance you are using.
    1. Researcher
    2. Learner
  4. Finally, you should be presented with a page offering you a number of profiles to choose from.

The reason there are 2 steps is so that I can more flexibly manage user access.

Quick Links

Learner hub

The learner.cs.sun.ac.za instance is for light coursework and testing. If you need more resources for research, you can request access to the other instances.

Please see the USER_README in the root of your storage for setup and examples.

Feel free to test your work here, or if for a course, you may well be given instructions by your lecturer on how to best make use of the environment.

Fast Facts
  • Link: https://learner.cs.sun.ac.za/
  • RAM: 1 to 4G per user
  • CPUs: 1 to 4 per user
  • GPU: 0 to 1 per user, which may also be shared with other users
  • Storage:
    • Home: 5G
    • Shared: 500G (All users have Read Only access)
  • User access: CS postgrads, some courses and on request

User server persistence and shutdown

When you logout, your server will be shutdown. All foreground and background tasks will be stopped. If you leave the the browser interface idle, or close the window, your server will be shutdown after an hour.

Login

After then first login step, you should be presented with a GitLab login page. Please follow the instructions on the GitLab login page to finish logging in.

Note

You will see your username in JupyterLab is "jovyan". This is a JupyterNotebook reference, which you are free to lookup.

I kept this name, as there is no reason to change it for the Learner JHub instance.

Storage

Yearly storage wipe

At the end of each year, I may destroy all the user-home volumes (everything under ~/) from this instance. So you are advised to backup your work with the expectation that it will be gone in January.

This applies to students and staff alike.

LeanerHub users are allocated 5Gigs of home storage, and have read only access to 500Gig of shared storage.

The shared storage is where lecturers can place large resources for students to have easy access to.

Researcher hub

The researcher.cs.sun.ac.za

Fast Facts
  • Link: https://researcher.cs.sun.ac.za/
  • RAM: 1 to 256 per user, host dependant
  • CPUs: 1 to 32 per user, host dependant
  • GPU: 0 to 1 per user, which may also be shared with other users
  • Storage:
    • Home: 25G (Can be increased on request with motivation)
    • Ephemeral Storage: 1G to 1T (This storage gets destroyed when your server stops)
    • /dev/shm: Makes up part of your RAM
  • User access: User access only granted on request. Negotiate this with your supervisor.

User server persistence and shutdown

Your server will NOT be shutdown in the background.

Therefore, please shutdown your instances when you are done with them.

This is important, as resources requested by your server will be unavailable for other users till your server is stopped.

Login

After the first login step you will need to login with your Uni credentials. This is simply your US username and password.

Storage

Yearly storage wipe

At the end of each year, I may destroy all the user-home volumes (everything under ~/) from this instance. So you are advised to backup your work with the expectation that it will be gone in January.

This applies to students and staff alike.

W.I.P

Last update: 2023-08-29
Created: 2023-08-29