How LLMs work
Open up large language models such as ChatGPT or Claude: how text is cut into tokens, how every next word is scored and drawn, how the model learns from text and is then taught to be an assistant. No programming or maths needed. Every number in the figures is real, recorded from a real small model.
- Beginner
- About 2 to 3 hours
- 11 sections
- 22 lessons
The course opens in batches, each with a limited number of free seats. You take your seat in the Academy, signing in with Google or GitHub.
Course outline
Section 1
A next-word machine
About this course
Lesson · 4 min
Guessing the next word
Lesson · 8 min
No lookup
Lesson · 8 min
Section 2
Tokens: how the machine reads
Text cut into pieces
Lesson · 8 min
Every machine cuts its own way
Lesson · 10 min
Section 3
Words as numbers
Every token is a list of numbers
Lesson · 8 min
What “near” means
Lesson · 8 min
Section 4
Attention: words looking at words
Every word looks back
Lesson · 10 min
Reading left to right
Lesson · 10 min
Section 5
Picking the next word
Scores become probabilities
Lesson · 8 min
The temperature knob
Lesson · 10 min
Section 6
Learning from text
Watch a model grow up
Lesson · 10 min
Is bigger better?
Lesson · 8 min
Section 7
From text writer to assistant
The base model
Lesson · 8 min
Teaching it to assist
Lesson · 10 min
Section 8
Why AI makes things up
Plausible is not true
Lesson · 6 min
What the machine can see
Lesson · 8 min
Section 9
Looking inside
Concepts inside the machine
Lesson · 8 min
Does the machine think ahead?
Lesson · 8 min
Section 10
One answer, start to finish
Inside an assistant that does things
Lesson · 10 min
Section 11
Final exam
Final exam
Quiz · 30 min
After the exam
Lesson · 5 min