This page goes with the slides of the session: Introduction to HPC (Google Slides).
What a cluster is#
High performance computing, HPC for short, means using many strong computers together. The systems at TU Dresden are run by ZIH, the university's computing centre. They all run Linux, and you use them from a terminal.
The most important thing to understand on day one: a cluster is not one big computer that you log in to and then use. It is hundreds of separate computers, called nodes, that many people share. You never start your work on them by hand. You describe what you need, and a scheduler finds a free place and runs it for you.
ssh takes you. Edit files, use git, send jobs. No heavy work here.The four parts in one sentence each
- Login node. The front door. Everybody shares it, so it is for light work only: editing,
git, looking at results, sending jobs. A training run on a login node slows down everyone and can get stopped. - Slurm. The batch system. You tell it how many cores, how much memory, how many GPUs and how much time you need. It puts your job in a queue and starts it when the resources are free.
- Compute node. One computer of the cluster. You do not log in to it. Slurm starts your program there.
- Job. One piece of work you hand to Slurm, usually as a small shell script with
#SBATCHlines at the top. Exercise 1 is exactly that.
Why a language model needs this#
Inside every layer of a language model, the main work is a matrix multiplication. A GPU is a processor that does thousands of these small multiplications at the same time. In exercise 3 you measure the difference yourself: the same multiplication, once on CPU cores and once on a GPU.
The second reason is memory. A model has to fit into the memory of the GPU. The small chat model from exercise 6 has 0.5 billion parameters and needs about 1 GB. The models people use at work are many times bigger. The GPUs here have 40 GB and 94 GB each.
The third reason is that research means running things many times. A job array starts the same script with four learning rates on four GPUs at once (exercise 5). On a laptop you would wait four times as long.
The five clusters#
ZIH runs five systems. Two are GPU clusters and those are the ones this lab uses. The numbers come from the hardware overview in the ZIH HPC Compendium (checked in October 2026). Hardware changes, so look there for the current state.
Alpha Centauri
GPUused in this labBuilt for AI and machine learning. Since 2021. We call it Alpha.
- Nodes
- 39
- GPUs
- 8 × NVIDIA A100 per node, 40 GB each. That is 312 GPUs.
- CPUs
- 2 × AMD EPYC 7352 per node, 48 cores
- Memory
- 1 TiB per node
login1.alpha.hpc.tu-dresden.de
login2.alpha.hpc.tu-dresden.de
Capella
GPUused in this labThe newer GPU cluster, also built for AI and machine learning. Since 2024.
- Nodes
- 156
- GPUs
- 4 × NVIDIA H100 per node, 94 GB each. That is 624 GPUs.
- CPUs
- 2 × AMD EPYC 9334 per node, 64 cores
- Memory
- 768 GiB per node
login1.capella.hpc.tu-dresden.de
login2.capella.hpc.tu-dresden.de
Barnard
CPUfine for exercises 1 and 2The big general purpose cluster. No GPUs. Since 2023.
- Nodes
- 720
- CPUs
- 2 × Intel Xeon Platinum 8470 per node, 104 cores. That is 74,880 cores.
- Memory
- 512 GiB per node
login1.barnard.hpc.tu-dresden.de
up to login4
Romeo
CPUA second general purpose CPU cluster. Since 2020. Not needed in this lab.
- Nodes
- 188
- CPUs
- 2 × AMD EPYC 7702 per node, 128 cores
- Memory
- 512 GiB per node
login1.romeo.hpc.tu-dresden.de
login2.romeo.hpc.tu-dresden.de
Julia
one huge machineNot a cluster of many nodes but a single computer with an enormous amount of memory, for programs that need everything in one place. Since 2021. Not needed in this lab.
- Nodes
- 1
- CPUs
- 32 × Intel Xeon Platinum 8276M, 896 cores
- Memory
- 47 TiB usable
julia.hpc.tu-dresden.de
A sixth system, the GPU cluster Deneb, is announced for 2027.
Every cluster has its own Slurm. A job can only be sent from a login node of the cluster it should run on. A job sent from Barnard never gets a GPU, because Barnard has none. So for the GPU exercises, log in to Alpha or Capella.
Which one should I take?
| You want to | Log in to | Why |
|---|---|---|
| do the whole lab in one place | Alpha or Capella | All seven exercises work there. |
| try the first two exercises | Alpha, or Barnard | They only need a CPU core. Alpha still wants --gres=gpu:1 in every job; on Barnard delete that line. |
| run a model that needs a lot of GPU memory | Capella | An H100 has 94 GB, an A100 has 40 GB. |
| test something quickly on a GPU | Alpha | It has a partition for short interactive tests: --partition=alpha-interactive. |
Where your files live#
There are two kinds of storage, and mixing them up is the most common beginner problem.
Your home folder
$HOME is permanent and backed up, and it is limited to 50 GiB. It is for your code and your settings. If pip and Hugging Face put their caches and models there, it fills up and you get Disk quota exceeded. With a full home folder you cannot submit jobs any more.
A workspace
A big, fast folder that you ask for with one command. It has an expiry date, and you can extend it. This is where the Python environment, the data, the models and the results of this lab go.
ws_allocate -F horse llm-lab 30 # prints /data/horse/ws/<zih-login>-llm-labYou do not have to type this yourself. The setup script creates the workspace llm-lab for 30 days and moves all caches into it. horse is the name of a file system. There are several, and they differ in how long a workspace may live:
| File system | Longest lifetime | Extensions | Kept after expiry | Where |
|---|---|---|---|---|
horse | 100 days | 10 | 30 days | all clusters. This lab uses it. |
walrus | 365 days | 2 | 60 days | all clusters |
cat | 30 days | 2 | 30 days | Capella, where it is the default |
quokka | 30 days | 2 | 30 days | Alpha and Romeo |
From the workspaces page of the Compendium, October 2026. The commands to list, extend and delete a workspace are on the cheat sheet.
A workspace ends. When the time is over the data is gone after the grace period. Copy what you want to keep, or extend the workspace in time with ws_extend. You can also ask for an e-mail reminder.
Three ways to work#
the normal way
A batch job
You write a job script and send it with sbatch. It runs when there is room, with or without you. The output goes to a log file. All exercises work like this.
for trying things
An interactive job
srun --pty bash -l gives you a terminal on a compute node. Good for installing, testing and debugging. The setup step uses it. Type exit to give the machine back.
for notebooks
JupyterHub
At jupyterhub.hpc.tu-dresden.de you start a notebook server on the cluster from the browser. The setup script adds a kernel called LLM Lab (venv-llm) for it.
What you need to get in#
- A ZIH login. The same user name and password you use for other university services.
- Membership in the course project. On the cluster, computing time belongs to projects. Ours is called
p_lv_llmsecrets, and every job script names it with--account. You get the link to join from the instructors in the session. A new membership needs some time before it works on all clusters, so join first and read on while you wait. - The university network or the VPN. The login nodes and JupyterHub cannot be reached from outside.
Then continue with Setup. More about access is in the Compendium.
Words you will hear#
| Word | What it means here |
|---|---|
| node | One computer of a cluster. |
| core | One processor core. You ask for them with --cpus-per-task. |
| job | One piece of work you hand to Slurm. It gets a number, the job ID. |
| job script | A shell script whose first lines start with #SBATCH and say what the job needs. |
| project, account | The group that the computing time is booked on. Ours is p_lv_llmsecrets. |
| partition | A group of nodes inside a cluster with its own limits. |
| walltime | The time limit of a job, as on a clock on the wall. When it is over, Slurm stops the job. |
| module | Software the cluster provides, switched on with module load. |
| workspace | A big folder with an expiry date. |