This is the second part of a series on how language models work, built from the ground up. The first part covered what a neural network is made of and how it learns: an office building of workers, managers and judges, trained by four steps on repeat. This one covers everything that part skipped on the way to language. How a piece of text becomes something the building can process, and how a word, once inside, reads the words around it.
The seven questions this answers
How does text turn into numbers? A worker multiplies. You can't multiply "king".
What does the model actually "see" when it reads a sentence?
Where does word meaning come from, given that nobody typed a dictionary into it?
How is the next word chosen, and then the one after it? Who decides, and from what?
How does "it" find the noun it refers to? And what are Q, K and V, the three arrows in every attention diagram?
Why does the first word of a reply take longer than the rest?
What does "heads" mean?
There are verified numbers throughout, and two small models you can paste and run. The first has a ten-word vocabulary and discovers, without being told, that "king" and "queen" are the same kind of thing. The second has twelve words and works out, from nothing but next-word prediction, that "it" refers to the subject of the sentence.
The five-minute recap
If you've read Part 1, skip this. If not, here is everything you need from it.
A neural network is a stack of layers. Each layer is a row of neurons, and a neuron does one thing: multiplies each input by a number (its weights), adds them up, adds one more number (its bias), and passes the total to an activation function, which for our purposes is max(0, x): negatives become zero, positives pass through. That function is called ReLU, and without it any number of layers collapses into a single one.
Part 1 pictured this as an office building. Facts arrive at the ground floor. Workers on each floor read the input, give points per fact, add their starting points, and each writes down one number. A manager at every elevator zeroes any negative number. What a floor produces becomes the next floor's input. A boss at the top makes the call.
Three words this article uses constantly, so here they are before anything else.
A memo is one token's row of numbers as it travels through the building. It's what a worker reads. A worker reads one memo at a time: one memo in, one number out.
A tray is the row of numbers the workers produce for one memo, one number per worker. Twelve workers, twelve numbers in the tray. If seven memos are on the floor, there are seven trays.
The judges replace the boss. There is one judge per word in the dictionary: ten judges for a ten-word toy, fifty thousand for a real model. Each judge reads the tray of the last memo and writes one score for their own word. The highest score is the next word. The building then glues that word onto the end of the text and runs again, and that loop is how every reply you've read from a chatbot was written, one word at a time. You'll see it run, with real numbers, in the middle of this article.
Training is a four-step loop. Forward pass: make a guess. Loss: measure how wrong it was. Backpropagation: walk backwards through every layer working out how much each weight influenced the error. Gradient descent: nudge every weight against that influence, a small step, all at once. Repeat until the loss stops falling, then freeze everything.
One rule from that loop matters more than the others here: a weight is any number the loop is allowed to change. Keep that in mind. The surprise in this article is which numbers turn out to be weights, and which numbers that are called weights aren't.
The problem
Every worker in the building multiplies numbers. You just handed it the word "king."
There's no arithmetic you can do on a word. So before anything reaches the ground floor, text has to become numbers, and it has to become numbers in a way that preserves meaning. king should end up close to queen and far from banana, or the workers will have nothing useful to detect.
That translation happens at three desks in the lobby. Each solves a different problem, and each has a way of going wrong.
Desk 1: Cutting text into pieces
The first decision is how big a piece of text to translate at a time. There are three obvious answers. Two of them fail.
Idea 1: one number per character
The simplest scheme imaginable. Assign every character an integer: H → 1, e → 2, l → 3, o → 4. So "Hello" becomes [1, 2, 3, 3, 4].
Why it fails. The word "understanding" is 13 characters, so it becomes 13 separate inputs. Every one of them is a full trip through the whole building. Thirteen forward passes to process one concept.
The cost is bad, but the deeper problem is that the model never receives the word. It receives u, then n, then d, and has to work out from scratch that "under" is a prefix and "stand" is a root. You're teaching someone to read by showing them letters and never a word. Enormous training capacity gets spent reconstructing basic vocabulary instead of learning anything above it.
Idea 2: one number per word
More intuitive. "The cat sat" → [1, 2, 3]. Each number is a whole concept.
Three separate problems sink it.
The form problem. Are cat, cats and cat's three unrelated concepts? Under this scheme, yes. Three different numbers, nothing connecting them. The model has to learn that they're related the hard way, separately, for every noun in the language.
Out of vocabulary. Someone types teh. It isn't in the list, so it isn't anything. It becomes an "unknown" token, and that word is now a hole in the sentence. The same happens to every name, every new product, every technical term coined after the list was built.
The vocabulary explosion. English has over 170,000 common words. Each one needs its own row of numbers in a lookup table, and at realistic row sizes that's roughly 3 billion parameters spent on the table alone, before a single layer of actual network. Add a second language and it multiplies.
Idea 3: subwords
Bigger than a character, smaller than a word.
Common words stay whole. Rare words break into recognisable pieces:
"understanding" → ["under", "stand", "ing"]
A vocabulary of 30,000 to 100,000 tokens can represent almost any text in any language. That's two orders of magnitude smaller than the word-level table, and nothing is ever unreadable, because anything not in the list falls back to smaller pieces.
But nobody sits down and decides which chunks make the list. An algorithm finds them.
How the chunks get chosen: byte pair encoding
BPE is four rules applied over and over:
Start with every character as its own token.
Find the most frequent adjacent pair in the training text.
Merge that pair into one new token.
Repeat until you reach the vocabulary size you want.
Let's run it on a tiny corpus. The word hug appears three times, pug twice, hugs once.
Starting point. Everything is split into characters:
h u g ×3
p u g ×2
h u g s ×1
Round 1. Count every adjacent pair across the whole corpus:
u+g = 6 h+u = 4 p+u = 2 g+s = 1
u+g wins. Merge it into a single token, ug:
h ug ×3
p ug ×2
h ug s ×1
Round 2. Recount. The pairs have changed, because ug is now one unit:
h+ug = 4 p+ug = 2 ug+s = 1
Merge h+ug into hug.
Round 3. p+ug is now the most frequent at 2. Merge into pug.
Round 4. hug+s at 1. Merge into hugs.
Final vocabulary: the five characters h u g p s, plus four merged tokens ug hug pug hugs.
Nobody told it that "hug" is a word or that "-s" marks a plural. It merged whatever was frequent, and in real language, frequency tracks meaning closely enough that common words end up as single tokens and rare ones stay in pieces. The vocabulary organises itself.
Pass example. A new word arrives that never appeared in training: pugs. It tokenizes cleanly as pug + s. The model has never seen the word and handles it anyway.
The fail-safe. Something alien like xyzzy falls back to individual characters. Ugly and inefficient, but never an unreadable hole. That's the property word-level tokenization couldn't give you.
What real tokenizers do
The splits GPT-style tokenizers produce surprise people:
| Input | Tokens | Why |
|---|---|---|
"Hello world" |
["Hello", " world"] |
The space attaches to the front of the next word |
"don't" |
["don", "'t"] |
Contractions split at the apostrophe |
"artificial intelligence" |
["art", "ificial", " intelligence"] |
Splits follow frequency, not syllables |
That last row is the one to stare at. "Artificial" splits as art + ificial, which is linguistically nonsense. BPE never learned linguistics. It learned which character pairs co-occur.
Why any of this matters to you
Context limits are token limits. A 128K context is 128,000 tokens, not words. English averages around 1.3 tokens per word. Code runs much higher, because punctuation and indentation eat tokens.
Billing is per token. Not per character, not per word.
It explains the strawberry problem. Ask a model to count the r's in "strawberry" and it often fails. It never saw letters. It saw two or three chunks with ID numbers attached. Asking it to count characters is like asking you to count the pixels in a word you're reading.
Languages aren't treated equally. The same sentence might be 10 tokens in English and 30 in Hindi or Thai, because the tokenizer was trained mostly on English text. That's three times the cost and three times the context consumed for identical meaning.
Desk 2: From arbitrary IDs to meaning
Tokenization hands you "Hello world" → [15496, 995].
Those numbers are meaningless. 15496 doesn't know it's a greeting. 995 has no connection to the Earth. They're catalogue numbers, assigned by whatever order the merges happened in, and 15497 is not a related word.
Hand those to a worker and he multiplies a catalogue number by a weight. Garbage in, garbage out.
The fix: a list of numbers per token
Instead of one arbitrary integer, each token becomes a list of numbers. That list is its embedding, and the rule that makes it useful is simple: tokens with similar meaning get similar lists.
The cleanest way to think about it is a survey. Every token answers the same 768 questions, each with a number roughly between −1 and +1:
"king" Q1: +0.31 Q2: −0.88 Q3: +0.12 … Q768: −0.44
"queen" Q1: +0.29 Q2: −0.85 Q3: +0.61 … Q768: −0.41
"banana" Q1: −0.77 Q2: +0.05 Q3: −0.23 … Q768: +0.90
Two tokens are similar if their answer sheets look alike. king and queen agree on most questions and differ sharply on a few. king and banana disagree almost everywhere.
That's the entire concept, and it needs no spatial imagination. You already understand "these two survey responses are similar."
The catch: nobody wrote the questions. Training invented all 768, and none of them has a name. There is no "gender" question at position 47. You cannot open the table, read column 12, and learn what it tracks. Meaning lives in the pattern across all 768, not in any single one.
What "768" is
It's just how long the list is. Every token in the vocabulary gets exactly that many numbers, so the whole embedding table is a grid: 100,000 rows × 768 columns.
Why not 2? Because you run out of room, and the way you run out is instructive. With two numbers per word you can place words on a flat sheet. Now try to satisfy all of these at once: dog near cat (pets), dog near wolf (canines), cat far from wolf, dog near puppy, dog near loyal. On a flat sheet you can arrange three or four of those before the fifth forces you to break an earlier one. Each extra dimension is another independent direction in which to be "close", and at 768 a word can be near hundreds of other words along hundreds of different axes without those relationships fighting.
Why not a million? Three costs. The table itself gets huge. More importantly, 768 isn't only the embedding width; it's the width of the pipe running through the entire model, so every layer's cost scales with it. And past a point, extra dimensions capture noise rather than structure.
So the number is an engineering compromise, chosen by whoever designed the model. BERT-base and GPT-2-small both used 768, and the convention stuck. GPT-3 used 12,288.
You cannot picture 768 dimensions, and neither can anyone else
There's no trick that unlocks it. Researchers who build these models are in exactly the same position. What they do instead is keep the answer-sheet picture, or picture 3D and remember which instincts transfer and which lie.
Transfers fine: distance (Pythagoras just keeps adding terms), "closer means more similar", directions being meaningful, adding vectors.
Breaks badly: your sense of how many perpendicular directions fit. In 3D you get three and then you're out. In 768 dimensions you get 768 exactly perpendicular directions, but if you accept nearly perpendicular, you get millions. Two random directions in high-dimensional space are almost always close to perpendicular, so they barely interfere with each other. That's why a few hundred numbers can hold far more independent concepts than you'd expect.
When you see a 3D scatter plot of word clusters, that's 768 dimensions squashed to three so a human can look at it. Like a shadow on a wall: real information about the shape, with depth thrown away. Clusters in those plots are meaningful. Distances between clusters usually aren't.
The classic picture, and its caveat
Every explainer shows this diagram, and it's worth showing because it's the clearest illustration anyone has produced of "meaning has geometry":
Because the man → woman arrow and the king → queen arrow are the same arrow, you can do arithmetic on meaning:
king − man + woman ≈ queen
Start at king, remove the maleness direction, add the femaleness direction, land near queen. Nobody labelled those axes. Training put words where they needed to be, and the structure fell out.
Now the caveat, which most explainers skip. That picture comes from word2vec, a 2013 system with one vector per whole word. It predates subword tokenization. In a modern model, grandfather is probably one token, so it has one row and the picture holds. But a rarer word might split into three fragments, and then there is no row for the whole word. There are three rows for three fragments, and Vy on its own means almost nothing.
So where does the meaning of a split word live? Nowhere at this desk. It gets assembled upstairs, by the layers that let fragments read each other. The embedding table is the starting point, not the finished representation. Every vector leaving Desk 2 is provisional, even for single-token words: bank leaves with its dictionary meaning, and only becomes river-bank or money-bank once the floors above have done their work.
Vector arithmetic on meaning didn't die with word2vec, incidentally. It moved upstairs. Researchers routinely find that directions in a model's middle layers encode concepts like formality or sentiment, and you can add those directions during generation to steer the output. That's king − man + woman grown up: same arithmetic, but on representations that have already read their context. You'll see the mechanism that does the reading in the second half of this article.
What physically happens at this desk
A lookup. The token ID is a row number:
ID 15496 → row 15496 → [0.12, −0.45, 0.88, …]
Nothing more. And this is the moment to remember the rule from the recap: the numbers in those rows are weights. They started as random noise. Training shaped them. You'll watch it happen in the code.
Desk 3: Injecting order
Here is what breaks without the third desk.
"The dog bit the man" "The man bit the dog"
Same words. Same embeddings. And to the arithmetic that follows, these two sentences are identical.
Why: the model sees a set, not a sequence
The tokens stay as separate vectors, one per position, all the way through. They're never summed into one. So it looks like order should survive: slot 0 holds dog, slot 3 holds man.
But the operations that follow (attention, which is the second half of this article) are permutation-equivariant: swap the input vectors and the outputs swap identically. Nothing inside the math ever references "slot 0." So what the model effectively receives is the set of vectors, and the two sentences above are the same set:
"the dog bit the man" → { the, dog, bit, the, man }
"the man bit the dog" → { the, man, bit, the, dog }
That property is called permutation invariance. One of those sentences is a headline and the other is an ordinary Tuesday, and the model can't tell which.
The fix: add a stamp to every position
Before anything else runs, add a small position-specific vector to each token's embedding. Not append. Add, number by number.
Now the at position 0 is a slightly different vector from the at position 1. The word's identity is still there, because its own numbers dominate. But it now carries where it sits as well as what it is, and the two sentences are different sets.
Why add, not concatenate
You could instead lay the embeddings side by side, so slots 0 to 767 are always the first word and 768 to 1535 always the second. That encodes position perfectly. It just doesn't scale, for two reasons.
Fixed window. A model built for two tokens can't read three. A thousand tokens of context would need 768,000 input slots.
No shared learning. A worker who learns "royalty in slots 0 to 767" knows nothing about royalty in slots 768 onward. Every pattern has to be relearned for every position.
Adding fixes both. The vector stays 768 wide no matter how long the sequence is, and a worker who learns to spot royalty spots it at any position, because the word's own numbers dominate and the stamp is a smaller nudge on top.
Why the stamps are sine waves
The original transformer built each stamp from sine and cosine waves at different frequencies. Each of the 768 slots follows a wave of a different wavelength, so every position gets a unique combination of values, like a fingerprint.
Three reasons that beats simply using the position number:
Nothing to learn. The stamps come from a formula, so there are no extra parameters and nothing to train.
Relative distance is computable. Because the waves are related mathematically, the offset between position 5 and position 12 has a consistent form. The model can learn "attend three words back" rather than memorising absolute slots.
It extrapolates. Waves are continuous, so the formula produces a valid stamp for position 5,000 even if training never went past 512.
Most current models have moved to a technique called RoPE, which rotates the vectors instead of adding to them. Same problem, better solution. The sine-wave version is where the idea started and it's the one that explains the reasoning.
Where the weights are
Now the question the recap set up. In a language model, which numbers can the training loop change?
The embedding table is weights. Every row, every one of the 768 numbers per token. It feels like reference data, but it isn't. Each row started as noise and got shaped by the loss, exactly like every weight in Part 1. In a small GPT the table is roughly a third of all parameters. It isn't a side component.
Every worker on every floor is weights, as before. And every judge's list at the top.
The dictionary is not. Token 4821 ↔ import was decided by a counting program before training started, and it's frozen forever. There's no gradient for it; it isn't even a number being multiplied, it's a lookup.
The sine-wave stamps are not. Computed from a formula. (Some models, GPT-2 among them, use a learned position table instead. Both designs exist.)
The text is not. Same rule as always: read thousands of times, never edited.
So the pipeline splits cleanly:
text → dictionary frozen
→ embedding table WEIGHTS
→ position stamp formula
→ the floors WEIGHTS
→ the judges WEIGHTS
Why this matters: backprop reaches the table
Take one training example: the corpus contains "long live the king", the model sees "long live the" and must predict the next token.
The forward pass looks up three rows, stamps them, sends them up. The judges score every word. The true answer is king; the loss measures how little probability it got.
Then the influence report walks down. Judges first. Then the floors. And then it keeps going, past floor 1, into the three embedding rows that were fetched. Each row gets the same question every weight gets: if you changed slightly, how much would the answer change?
Every weight steps against its influence, all at once. The judges' lists shift. Every floor shifts. And those three rows shift, toward whatever makes king more likely after that context.
The other 99,997 rows do not move. banana wasn't in this sentence, contributed nothing, has zero influence, gets no correction. That's the same sparsity rule from Part 1, applied to the lookup table.
Where embedding geometry comes from
That last paragraph is the whole mechanism.
A row only updates when its token appears. So each row is shaped exclusively by the contexts that word shows up in.
king and queen appear in overlapping contexts, so across billions of sentences they receive similar nudges and drift toward similar values. banana gets nudged by an entirely different set of sentences and ends up somewhere else. Not pushed away; just never pulled in.
Nobody specifies that royalty exists. The only instruction is "predict the next token, here's how wrong you were." Geometry encoding gender, tense, formality and sentiment falls out as a side effect, because organising words that way is what makes next-token prediction accurate. The old linguistics line for it: you shall know a word by the company it keeps.
Build it: a language model in 60 lines
Here is the smallest complete language model that still has every piece: a dictionary, an embedding table, one floor of workers with a manager, and a row of judges. No frameworks. Paste it into a .py file and run it.
The task: read two words, predict the third.
import numpy as np
np.random.seed(0)
# --- the dictionary: built before training, never changes ---
words = ["long","live","the","king","queen","said","yes","peel","banana","now"]
stoi = {w:i for i,w in enumerate(words)}
V = len(words)
corpus = ["long live the king", "long live the queen",
"the king said yes", "the queen said yes",
"peel the banana now"]
# context = two previous tokens -> predict the next one
X, Y = [], []
for s in corpus:
ids = [stoi[w] for w in s.split()]
for i in range(len(ids) - 2):
X.append([ids[i], ids[i+1]]); Y.append(ids[i+2])
X, Y = np.array(X), np.array(Y)
D, H = 4, 12 # embedding width, workers
E = np.random.randn(V, D) * 0.3 # EMBEDDING TABLE (weights)
W1 = np.random.randn(2*D, H) * 0.3; b1 = np.zeros(H) # workers (weights)
W2 = np.random.randn(H, V) * 0.3; b2 = np.zeros(V) # judges (weights)
def similarity(a, b):
return E[stoi[a]] @ E[stoi[b]] / (np.linalg.norm(E[stoi[a]]) * np.linalg.norm(E[stoi[b]]))
print(f"before: king~queen {similarity('king','queen'):+.3f} king~banana {similarity('king','banana'):+.3f}")
lr = 0.5
for step in range(3001):
# 1. FORWARD
emb = E[X].reshape(len(X), 2*D) # fetch two rows, lay side by side
z1 = emb @ W1 + b1 # workers score
a1 = np.maximum(0, z1) # the manager
logits = a1 @ W2 + b2 # one score per word
p = np.exp(logits - logits.max(1, keepdims=True)); p /= p.sum(1, keepdims=True)
# 2. LOSS (cross-entropy: -log of the probability given to the right word)
loss = -np.log(p[np.arange(len(Y)), Y]).mean()
# 3. THE REPORT walks down: judges -> workers -> and into the embedding rows
d = p.copy(); d[np.arange(len(Y)), Y] -= 1; d /= len(Y)
gW2 = a1.T @ d; gb2 = d.sum(0)
d1 = (d @ W2.T) * (z1 > 0) # silent workers get zero
gW1 = emb.T @ d1; gb1 = d1.sum(0)
gE = np.zeros_like(E)
dE = (d1 @ W1.T).reshape(len(X), 2, D)
for i, (a, b) in enumerate(X): # only rows that appeared get a nudge
gE[a] += dE[i, 0]; gE[b] += dE[i, 1]
# 4. EVERYONE ADJUSTS, including the table
W2 -= lr*gW2; b2 -= lr*gb2; W1 -= lr*gW1; b1 -= lr*gb1; E -= lr*gE
if step in (0, 300, 3000): print(f" step {step:4d} loss {loss:.3f}")
print(f"after: king~queen {similarity('king','queen'):+.3f} king~banana {similarity('king','banana'):+.3f}")
print("\npredictions:")
for ctx in (["live","the"], ["the","king"], ["the","queen"], ["peel","the"]):
e = E[[stoi[c] for c in ctx]].reshape(1, -1)
lg = np.maximum(0, e @ W1 + b1) @ W2 + b2
pr = np.exp(lg - lg.max()); pr /= pr.sum()
top = pr[0].argsort()[::-1][:2]
print(f" {' '.join(ctx):10} -> " + ", ".join(f"{words[t]} {pr[0][t]*100:.0f}%" for t in top))
Three things to notice in the code before running it.
stoi is the dictionary. It's a plain Python dict, built once at the top, never touched by the loop. That's Desk 1.
E is the embedding table, and it's in the update line at the bottom next to W1 and W2. That's the whole point of the first half of this article in one line of code.
The gE loop is sparsity on the table: only the rows that appeared in this batch get a gradient. Everyone else's row gets nothing.
A note on what's missing: this toy lays the two context vectors side by side rather than adding a position stamp, and it has no attention. That's deliberate; both arrive in the second model, below. What's here is Desks 1 and 2 wired into the building from Part 1.
What you get
before: king~queen +0.577 king~banana -0.487
step 0 loss 2.276
step 300 loss 0.142
step 3000 loss 0.139
after: king~queen +0.988 king~banana -0.458
predictions:
live the -> king 52%, queen 48%
the king -> said 100%, now 0%
the queen -> said 100%, now 0%
peel the -> banana 100%, now 0%
The predictions are correct, and the live the line is the model being correctly uncertain: both king and queen follow that context in the corpus, so it splits almost exactly in half. It didn't memorise one and forget the other.
But the predictions aren't the interesting part.
Look at the two similarity lines. Before training, king and queen were two random rows with a coincidental similarity of 0.577. After 3,000 rounds they sit at 0.988, which is nearly the same vector. king and banana didn't move toward each other at all.
Nobody wrote a rule about royalty. The corpus contains the king said and the queen said, so both rows received similar nudges every round, and drifting together was what lowered the loss. That is the entire origin of embedding geometry, visible in a ten-word model. The starting loss, 2.276, is almost exactly ln(10), which is what pure guessing among ten words gives you.
The worker's equation, on real tokens
Part 1 showed a worker computing 0.5 + (2 × 1) = 2.5 on a checkbox. The equation is unchanged here. What's changed is what x contains.
Feed the trained model "the king". The dictionary gives [2, 3]. The table gives two rows of four numbers each, laid side by side into eight:
[-1.118, 0.875, -1.337, 1.314, 0.934, -0.506, 0.768, -0.523]
└───────── "the" ─────────┘└──────── "king" ────────┘
Not readable by you. Nobody can look at −1.118 and say what it means. But a worker doesn't need to. He multiplies.
Worker 0 has eight weights, one per slot, and one bias:
weights: [-1.061, 0.243, -0.187, -0.534, -1.278, 0.731, -0.201, -0.260]
bias: +0.739
Slot by slot:
-1.118 × -1.061 = +1.186
0.875 × 0.243 = +0.213
-1.337 × -0.187 = +0.250
1.314 × -0.534 = -0.702
0.934 × -1.278 = -1.193
-0.506 × 0.731 = -0.370
0.768 × -0.201 = -0.155
-0.523 × -0.260 = +0.136
───────
-0.636 the weighted sum
+ bias +0.739
───────
+0.103 his score
Manager: max(0, +0.103) = 0.103. Positive, so he speaks. Barely.
Points per slot, add them up, add starting points, hand to the manager. Identical to the action worker in Part 1, except there are eight slots instead of one and the slots have no human-readable meaning.
That last part deserves a sentence. The action worker responded to a checkbox that meant something. These workers respond to patterns across embedding slots: combinations of coordinates that have no name. Worker 0 is negative on slots 1 and 5, positive on slot 6. That isn't "likes action." It's a direction in meaning-space, and he fires for whatever tokens point that way. The workers detect regions of meaning, not words.
Sparsity, measured
Here is what the manager does across all twelve workers for three different inputs:
the king [0.1 4.0 5.0 0 0 0 3.1 0 0.1 2.5 1.5 0 ] 8 awake
the queen [0.1 4.1 5.1 0 0 0 3.2 0 0.1 2.6 1.5 0 ] 7 awake
peel the [3.3 0 0 0 0 4.2 0 0 0 0 0 0 ] 2 awake
Three things fall out of that table.
peel the silences ten of twelve workers. A completely different circuit lights up: workers 0 and 5 only. The royalty workers contribute exactly zero. That's Part 1's biology-worker-on-a-code-prompt effect, at a scale you can count.
the king and the queen produce nearly identical trays: 4.0 versus 4.1, 5.0 versus 5.1, the same workers awake at the same volumes. Which is why both predict said at 100%. The judges receive virtually the same input either way.
Worker 1 shouts 4.0 for royalty and is silenced at −3.1 for fruit. Not dead. Conditionally active, exactly as in Part 1.
And the manager is doing the same load-bearing job as before. Delete him and this floor folds into the judges' layer, and the model becomes a single points-per-slot scorer over embedding coordinates. It could never learn that a combination of slots means royalty, only that each slot is worth a fixed amount. Under sigmoid, those ten zeros in the peel the row would be 0.3, 0.5, 0.2: every irrelevant worker contributing noise to every judge on every input.
From tray to word: the judges
The tray is where Part 1 handed things to a boss. Here it's handed to ten judges, and it's worth slowing down, because this is where a row of numbers becomes a word.
A judge is a worker one floor up. He takes the tray, twelve numbers, and has twelve weights of his own, one per slot. Points per slot, add them up, add his bias. That's his score. Here is judge said reading the the king tray:
tray his weight
0.10 × -0.47 = -0.05 (worker 0)
4.03 × +1.72 = +6.93 (worker 1)
4.96 × +0.66 = +3.27 (worker 2)
0.02 × -0.40 = -0.01 (worker 4)
3.13 × +0.54 = +1.69 (worker 6)
0.10 × -0.03 = -0.00 (worker 8)
2.54 × +0.43 = +1.09 (worker 9)
1.45 × +0.23 = +0.33 (worker 10)
four silent workers × anything = 0
───────
+13.24
+ bias -0.14
───────
+13.10 judge "said"
Half of that score is one line: worker 1 shouting 4.03, times judge said's weight of 1.72. Training taught judge said to trust the royalty worker, because in the corpus said follows royalty. Judge banana reads the same tray with his own twelve weights, has −0.11 on that slot instead, and comes out at −3.94. Same twelve numbers, opposite verdicts, because the weights differ. That's the entire mechanism of a judge.
Then the vote. Ten scores side by side:
said +13.10 now +3.05 the +1.98 long -0.68 yes -0.91
live -1.94 peel -1.97 king -3.94 banana -3.94 queen -5.15
Highest wins. Softmax turns the gap into probabilities, and said at +13.1 against a runner-up at +3.1 comes out at 99.99%. That's the said 100% in the output block above, with the arithmetic shown.
Three things to hold onto.
A judge never sees the words. He sees twelve numbers and grades how much they look like "the kind of tray my word comes after". the king and the queen produce nearly identical trays, so every judge gives nearly identical scores, so both get said.
There is one judge per dictionary entry, no exceptions. Ten words, ten judges. GPT-2's dictionary has 50,257 tokens, so 50,257 judges: one for said, one for the with the leading space, one for ing, one for ., one for <|endoftext|>. Every one of them reads the last memo and scores, every single time a word is chosen. Formally this layer is the unembedding or LM head, and since each judge is as wide as the memo (768 weights in GPT-2), the whole thing is a 768 × 50,257 grid. The vote is one matrix multiply, which is why a huge dictionary barely costs any time.
In many models the judges are the embedding table. Look at the shapes: Desk 2's table is 50,257 rows of 768; the judges are 50,257 columns of 768. GPT-2 uses the same numbers for both, a trick called weight tying. A token's embedding row doubles as its judge's weights, so judge said scores a memo by how much it points in the same direction as the said embedding. "Does this memo look like said?" and "what does said look like?" turn out to be the same question asked from both ends.
Run it as a generator
Now the loop. Start with long live, take the winning judge, glue the word on, run again:
step 1 long live -> judges: the 100% -> pick "the"
step 2 live the -> judges: king 52%, queen 48% -> pick "king"
step 3 the king -> judges: said 100% -> pick "said"
step 4 king said -> judges: yes 100% -> pick "yes"
output: long live the king said yes
Three things in that run.
The model returns scores, not a word. Choosing is done by the loop around the model, and the model is stateless: it takes a list of IDs, returns ten scores, and forgets everything. The loop is what remembers, by keeping the list and appending one ID per step. That's ids.append(next) in any generation code you'll ever read.
Step 2 is a real fork. king at 52%, queen at 48%. Taking the top score every time (greedy) always gives king. Rolling a weighted die instead (sampling) gives queen roughly half the time, and the output becomes long live the queen said yes. That is the only reason the same prompt to a chatbot gives different wordings on different tries. Temperature is a knob on the die: divide the scores by T before softmax, and at T = 0.5 the split sharpens to 55/45, at T = 2 it flattens to 50/48 and banana creeps up to half a percent. High temperature doesn't add creativity; it makes the die fairer to bad options.
It stops when the loop says so. This toy has no end token, so we stopped at four words. A real dictionary includes <|endoftext|>, which sat at the end of every document during training, so its judge learned when "the end" is likely. When that judge wins, the loop exits. The other exit is a hard cap on length, which is the loop giving up, not the model finishing. That's why a reply can cut off mid-sentence.
And one limit to notice, because the second half of the article exists to remove it: the memo is always the last two tokens. At step 4 the model has forgotten long live the. It can never use anything further back than that window. A real model reads the whole context, and the mechanism that lets it is the reading room.
The pipeline so far
raw text
↓ tokenize cut into subword chunks, look up IDs frozen
[15496, 995]
↓ embed each ID becomes a row of 768 numbers weights
[0.12, −0.45, …]
↓ add position a stamp is added: order formula
[0.31, −0.29, …]
↓
the floors workers and managers, one memo at a time weights
↓
the judges one per word, read the last memo, highest wins weights
↓ glue the word on, run again
Each token now knows what it is and where it sits. Every vector is still context-free, though: bank has its dictionary meaning, not its meaning in this sentence, and no token has yet looked at any other.
That's the job of self-attention, the mechanism that lets each token decide which other tokens to read before the floors do their work. It's why bank next to river ends up different from bank next to savings, and it's the T in GPT. It's also what turns the toy above, which reads exactly two words, into something that reads a whole document.
That's the second half of this article.
The problem upstairs
Here is the example that runs through the rest of the article. Two sentences, one word swapped:
the dog saw the cat and it ___ → barked
the cat saw the dog and it ___ → purred
The word in the blank depends on what "it" refers to, and in this article "it" always refers to the subject: whichever animal came first.
Now look at what the lobby hands the first floor at the last position. In both sentences it's the same token, it, with the same stamp, position 6. The identical vector, number for number. Whatever the floors do with that vector, they do it identically in both sentences, so on its own the last position cannot tell barked from purred. Not with more workers, not with more floors.
The information is in the sentence. It's just sitting in other tokens' vectors. Something has to carry it across.
Idea 1: read the last few words
This is what the first model did: fetch two rows, lay them side by side, send the pair up.
Fail. The two words before the blank are and it in both sentences. In fact every one of the eighteen sentences in this article's training corpus ends with and it, so a two-word window sees the same input eighteen times, and the best it can do is split its guess three ways.
Widen the window to seven words and you're back to the concatenation problem from Desk 3: a fixed window that can't read an eighth word, and a worker who learned "animal in slots 8 to 15" knowing nothing about an animal in slots 16 to 23.
Idea 2: average everything
Add up every vector in the sentence and divide by the count. Any length works, and every word contributes.
Fail, and provably. Addition doesn't care about order. the dog saw the cat and the cat saw the dog contain the same five vectors, so they have the same average, so the floors produce the same answer, so one of the two sentences is wrong. The position stamps don't rescue it, because they get averaged too: dog + stamp₁ + cat + stamp₄ adds up to exactly the same numbers as cat + stamp₁ + dog + stamp₄. The pairing between a word and its position is precisely what averaging throws away.
(The code at the end lets you try this. Loss stalls at 0.467 and every two-animal sentence comes out close to 50/50.)
What's actually needed
A weighted average, where the weights are chosen fresh for every sentence, by the sentence.
In sentence one, it should take almost all of its blend from dog and nearly nothing from cat. In sentence two, the reverse. Those weights can't live in the parameter file, because the file is the same for both sentences. They have to be computed from the tokens themselves, on the spot.
That is the whole idea of attention. Everything below is the machinery for computing those weights.
The reading room
Every floor of the building gets a new room, in front of the workers' room. Memos come up the elevator, go through the reading room first, and only then reach the workers' desks.
In the reading room, every memo reads every other memo before anyone scores anything. Here's the procedure.
Three stamps
When a memo arrives, three rubber stamps are pressed onto it. Each stamp is a grid of weights, and each turns the memo's numbers into a new list of numbers:
A request slip: what am I looking for? Formally the query,
q.A folder label: what do I contain, for anyone who's searching? Formally the key,
k.The contents: what I hand over if someone picks me. Formally the value,
v.
The three stamps are three weight matrices, W_Q, W_K, W_V. In the toy they're each 8 by 8. Every memo on the floor gets all three, from the same three pads.
The names come from databases. A database holds keys, you hand it a query, and it returns the value filed under the matching key. The reading room does the same thing with one difference, which is the point of the next few sections: it never picks one match. It takes a bit of every value, in proportion to how well each key matched.
Why three stamps, not one
The obvious shortcut is to skip the stamps and let a memo hold itself up against the other memos. Then "what am I looking for" would just be the memo.
What breaks: the best match for any memo is always itself, and the next best are the words most like it. it would go looking for other pronouns. bank would find savings, which happens to be useful, but only by luck. A pronoun's entire job is to find something unlike itself: a noun, a subject, a thing. The request-slip stamp is what lets "what I'm looking for" be different from "what I am".
The same goes for the other two. What a memo advertises on its label isn't what it hands over. dog at the front of a sentence should advertise subject noun, early, and hand over dogness. Three separate stamps let training shape those three roles independently.
The lookup, step by step
Take sentence one and follow the last memo, it, through the room. Every number below is from the trained toy at the end of the article.
Step 1: stamp everyone. Seven memos, seven labels, seven sets of contents. The last memo also writes a request slip:
q = [ 2.845 1.746 -0.933 2.493 1.712 3.703 -2.519 -2.134 ]
Eight numbers. Not readable, as usual. But watch what they do.
Step 2: score every label against the slip. Hold the slip up to a label and compute a match score. The score is a dot product: multiply slot by slot, add it all up. Here's the slip against dog's label:
slip label(dog)
2.845 × -0.433 = -1.232
1.746 × 0.338 = +0.590
-0.933 × 0.120 = -0.112
2.493 × -0.252 = -0.628
1.712 × 0.256 = +0.438
3.703 × 0.932 = +3.451
-2.519 × -0.555 = +1.398
-2.134 × 0.545 = -1.163
───────
+2.742
Stop and look at that block. It's the worker's equation from Part 1: points per slot, added up. But there's a difference, and it carries the whole article. The points aren't weights from the file. They're another memo's request slip. A worker's points are fixed at the freeze and identical for every input. This slip was written moments ago, for this sentence, by the token it. In the next sentence it writes a different slip, so the same label scores differently. That's what lets the weighting change from sentence to sentence, and nothing in the building so far could do it.
Do that for all seven labels:
the dog saw the cat and it
-19.1 +2.7 -12.0 -19.0 -11.5 -23.8 -24.9
Step 3: scale. Divide every score by √8 ≈ 2.83. (The reason is a section of its own, below.)
-6.74 +0.97 -4.24 -6.72 -4.07 -8.42 -8.81
Step 4: shares. Apply softmax: raise e to each score, divide by the total. Out come shares that add up to 1.
0.000 0.987 0.005 0.000 0.006 0.000 0.000
it gives 98.7% of its attention to dog, half a percent each to saw and cat, and effectively nothing to the rest.
Step 5: take the blend. Multiply each memo's contents by its share and add. Slot 0 of eight:
0.987 × -1.503 (dog) = -1.484
0.005 × -1.222 (saw) = -0.007
0.006 × +1.454 (cat) = +0.009
rest ≈ 0
───────
-1.482
Same arithmetic for the other seven slots. The blend is, to three decimals, dog's contents.
Step 6: add it to your own memo. it's own slot 0 was −0.134. Add the blend and it becomes −1.615. The memo that leaves the room is it + (mostly dog). It's still it, but it now carries dog with it, and that's what the workers see.
The same slip, the other sentence
Now sentence two. The last memo is the same token at the same position, so its request slip is identical, all eight numbers. Only the labels around it have changed.
sentence 1 the dog saw the cat and it
raw score -19.1 +2.7 -12.0 -19.0 -11.5 -23.8 -24.9
share .000 .987 .005 .000 .006 .000 .000
sentence 2 the cat saw the dog and it
raw score -19.1 +0.7 -12.0 -19.0 -9.5 -23.8 -24.9
share .001 .961 .011 .001 .026 .000 .000
dog scored +2.7 in position 1 and −9.5 in position 4. Same word, same slip, different stamp. And the blend it takes away is now 96% cat, so it leaves the room as it + (mostly cat):
entering the room, both sentences: [-0.134 1.050 1.381 …]
leaving, sentence 1 (dog): [-1.615 0.792 2.536 …]
leaving, sentence 2 (cat): [ 0.105 1.376 0.735 …]
That's the problem from the top of the article, solved. Two identical vectors went in. Two different ones came out. The difference is exactly the information that was sitting in the other tokens.
What the slip learned
Because the stamp was added to the embedding at Desk 3, and a dot product is multiply-and-add, every score splits cleanly into two parts: a word part and a position part. Here is what it's slip says to each word ignoring position, and to each position ignoring the word:
word part position part
bird +10.7 position 1 -5.9
dog +8.7 position 2 -5.7
cat +6.7 position 4 -18.1
and -3.6 position 5 -20.2
saw -6.3
the -7.7
it -8.9
dog in position 1 is 8.7 − 5.9 = +2.7. cat in position 4 is 6.7 − 18.1 = −11.5. The slip is asking two things at once: is it an animal, and is it early? The three animals are the only positive words. Positions 4 and 5 are punished about three times harder than 1 and 2.
Nobody wrote "subject" anywhere. The corpus has it referring to the first animal, the loss punished every wrong guess, and the slip drifted into the two questions that make the guess right. It's the both-detector from Part 1 and the king–queen similarity from the first model, one level up.
The formula
Here is the line from the 2017 paper, Attention Is All You Need. Every symbol in it is one of the six steps above.
Attention(Q, K, V) = softmax( Q Kᵀ / √dₖ ) V
| Piece | Reads as | Step |
|---|---|---|
Q |
every memo's request slip, stacked as rows | 1 |
K |
every memo's folder label, stacked as rows | 1 |
V |
every memo's contents, stacked as rows | 1 |
Q Kᵀ |
every slip scored against every label: a grid of dot products | 2 |
/ √dₖ |
divide every score by the square root of the slip's width | 3 |
softmax( … ) |
turn each row of scores into shares that add to 1 | 4 |
… V |
each row's shares times the contents, added up: the blend | 5 |
Step 6, the add, is outside the formula. It's called the residual connection, and in the code it's the line h = x[-1] + w @ Vv.
Shapes. With T tokens and slips dₖ numbers wide, Q is T × dₖ, Kᵀ is dₖ × T, and Q Kᵀ is T × T: one score for every pair of memos. For a 1,000-token input that's a million scores per room per floor. It's why attention's cost grows with the square of the context length.
In the toy only one row of Q is used, the last memo's. More on that shortly.
Why divide by √dₖ
A dot product adds up dₖ products. With slips 64 numbers wide, the typical score between two random ones is about 8; at 768 wide, about 28. Now look at what softmax does with scores spread across tens:
softmax([ 3, 1, 0]) → [0.844 0.114 0.042]
softmax([30, 10, 0]) → [1.000 0.000 0.000]
The second row is a hard lookup: one memo gets everything, the rest get shares of exactly zero. Zero share means zero contribution, which is fine on its own. But it also means zero influence when the report comes back down: the room can never be corrected toward a memo it's currently ignoring. It's the flat-at-the-edges problem from sigmoid in Part 1, and it hits on step one, before anything has been learned.
Dividing by √dₖ brings the typical score back to around ±1 regardless of width, so early in training every memo gets a share worth correcting. You can see the un-divided version in the toy: the raw scores after training are +2.7 against −19 and −24, and softmax of those is a hard 1.000 / 0.000. The trained room wants to be that decisive. Scaling makes sure it starts out soft enough to learn.
Soft lookup
One more word on softmax, because "attention weights" is the phrase you'll read everywhere. A database does a hard lookup: exact key match, one value back, everything else ignored. The reading room does a soft lookup: every key gets a score, and every value comes back in proportion. it in sentence one takes 98.7% dog, 0.5% saw, 0.6% cat. That 0.6% is real information (there's a cat in this sentence too), and a hard lookup would have thrown it away.
Every memo does this at once
I followed one memo through the room. In a real model every memo does the whole procedure simultaneously, and the result is a grid: one row per reader, one column per memo read, a share in every cell.
That grid is the attention matrix, and it's what the heatmaps in every transformer explainer show. A dark cell means "this reader took a lot from that memo".
Two things about it.
The toy computes only the bottom row. With a single floor, it is the only memo the judges ever hear, so its reading is the only one that matters. Stack a second floor and every row matters: the memo dog hands upward is read by the second floor's room, so dog needs its own reading too. Real models compute the full grid; the toy skips the other rows because nothing downstream would read them.
In a GPT-style model, half the grid is blanked out. Each memo may only read memos before it. The ones after are forced to a share of zero. The reason is generation: when the model is writing, the tokens after the current one don't exist yet, so training has to work under the same rule. That's the causal mask, and it's why these models are called decoders. It's also what makes the cache in a later section possible.
What real grids look like, once trained: researchers routinely find rooms whose grid is dark just below the diagonal (each memo reading the one just before it), rooms that link pronouns to their nouns, rooms that link a verb to its subject, and rooms that mostly stare at the first token when there's nothing useful to read. Nobody assigns those jobs. Same as the workers.
What leaves the room
Desk 2 came with a warning: every vector leaving it is provisional. bank leaves with its dictionary meaning and only becomes river-bank or money-bank once the floors above have done their work.
This is that work. The reading room adds (mostly river) or (mostly loan) to bank's memo, and the vector that reaches the workers is the context-aware one. Desk 2 also said vector arithmetic on meaning "moved upstairs": king − man + woman was adding a direction to a vector. The reading room adds a blend of other tokens' contents to a vector. Same operation, done by the model itself, on every token, on every floor.
Seven memos, one worker at a time
Here is the spot where the picture usually goes wrong, so let me be exact about it.
Seven memos leave the reading room. They do not go to the workers as a group. Each memo makes its own trip: dog goes through the twelve workers and comes out as dog's tray; cat goes through the same twelve workers and comes out as cat's tray; and so on, seven trips, seven trays. A worker never has two memos in front of him. Worker 3 reads dog and writes one number, reads cat and writes one number, reads it and writes one number, and each number lands in a different tray, the tray of the memo he was reading at the time.
The seven trips are independent, so a computer runs them simultaneously. That is all "the transformer processes tokens in parallel" means. It does not mean a worker looks at seven tokens together. It means seven separate one-memo jobs happen at the same instant. The reading room is the only place a memo learns anything about another memo.
And what happens to the trays that aren't the last one? They go up. Every memo's tray becomes that memo's rewritten version, and all seven ride to floor 2 together, where it reads dog again in the next reading room, but now a dog that has itself already read the. Twelve floors, twelve readings, each of a more worked-over dog. Only at the very top, and only during generation, do the six earlier trays get dropped: the judges read the last memo because that's the only prediction the loop asked for. During training, all seven go to the judges and all seven get graded.
The workers themselves behave exactly as in the first model. Here are it's trays for the two sentences:
it + (mostly dog) [0 0 4.4 0 8.7 0 0 0 0 5.7 0 4.7] 4 awake
it + (mostly cat) [0 0 7.1 0 0 0 0.03 0 0 9.1 0 3.8] 4 awake
Eight of twelve workers silent either way. The two awake sets overlap but differ, and the volumes differ, which is why the judges say barked for one and purred for the other. The room moved it into a different region of meaning, and the workers detect regions.
Why add, not replace
Step 6 adds the blend to the memo instead of replacing the memo with it. Two reasons, one for now and one for the next article.
For now: the memo has to keep being itself. it must remain a pronoun in position 6, the thing being described, so the workers can tell it apart from the describer. bank must remain bank. If the room replaced each memo with its blend, every token would dissolve into its neighbours, and a few floors up nothing would remember what it started as. Adding keeps the identity and layers context on top.
The other reason is that the add gives the influence report a straight path down through a hundred floors, which is a different problem. Next time.
Where the weights are, updated
Time to update the list from the first half.
Trainable: the embedding table, every worker, every judge, and now the three stamps in every reading room (plus a fourth, once there are several rooms, which you'll meet below).
Not trainable: the dictionary, the position-stamp formula, the √dₖ, and softmax. All formulas. Nothing to nudge.
And here is the trap. The shares are called attention weights in every paper you'll ever read, and by the rule from Part 1 they are not weights. They aren't numbers the loop changes. They're computed fresh for every sentence, like a worker's memo, and thrown away afterwards. Nothing in the parameter file says "it → dog". What the file stores is three stamps that, given this sentence, produce a slip and labels that make dog score highest. Hand the same stamps a different sentence and they produce different shares.
If you remember one thing from this section: attention weights are activations. The stamps are the parameters.
The report reaches the stamps
When the loss comes back down, it walks: judges → workers → the memo that left the room. There it splits, because that memo was own memo + blend. One branch goes straight into the token's embedding row. The other goes into the blend, and from the blend into the contents and the shares, through softmax into the scores, and from the scores into the slip and every label, which means into all three stamps and, again, into the embedding rows.
The shares get an influence number ("if dog's share were a bit bigger, how much better would the answer be?") but no update, because there's nothing to update. The influence flows through them into the stamps. Nudge the stamps, and next round the shares come out different. In the code this is the block under 3. THE REPORT, and it's the longest block for exactly this reason: every stamp gets its own line.
Build it: attention in 80 lines
The first model with two additions: the position stamp from Desk 3, and one reading room in front of the workers. Still no frameworks. The task: read a sentence ending in it, predict the next word.
The corpus is eighteen sentences built from a rule: it is the subject, and the subject makes its sound. Twelve of the eighteen contain a second animal as a distractor. Some start with today so the subject's position moves.
import numpy as np
np.random.seed(5)
# --- the dictionary: built before training, never changes ---
words = ["the","cat","dog","bird","saw","moved","today","and","it","purred","barked","chirped"]
stoi = {w:i for i,w in enumerate(words)}
V = len(words)
# --- the corpus: "it" always refers to the subject, the FIRST animal ---
sound = {"cat":"purred", "dog":"barked", "bird":"chirped"}
sents = []
for a in sound:
sents += [f"the {a} moved and it", f"today the {a} moved and it"]
sents += [f"the {a} saw the {b} and it" for b in sound if b != a]
sents += [f"today the {a} saw the {b} and it" for b in sound if b != a]
data = [([stoi[w] for w in s.split()], stoi[sound[next(w for w in s.split() if w in sound)]]) for s in sents]
D, H = 8, 12 # vector width, workers
pos = np.arange(8)[:,None] / 10000**(np.arange(0, D, 2)/D)
PE = np.zeros((8, D)); PE[:,0::2] = np.sin(pos); PE[:,1::2] = np.cos(pos) # position stamps (formula)
E = np.random.randn(V, D) * 0.5 # embedding table (weights)
WQ = np.random.randn(D, D) * 0.3 # request-slip stamp (weights)
WK = np.random.randn(D, D) * 0.3 # folder-label stamp (weights)
WV = np.random.randn(D, D) * 0.3 # contents stamp (weights)
W1 = np.random.randn(D, H) * 0.3; b1 = np.zeros(H) # workers (weights)
W2 = np.random.randn(H, V) * 0.3; b2 = np.zeros(V) # judges (weights)
def forward(ids):
x = E[ids] + PE[:len(ids)] # what each token is + where it sits
K = x @ WK; Vv = x @ WV # every memo: a folder label, and its contents
q = x[-1] @ WQ # the last memo writes its request slip
s = K @ q / np.sqrt(D) # slip against every label -> one score each
w = np.exp(s - s.max()); w /= w.sum() # scores -> shares that add up to 1
h = x[-1] + w @ Vv # take the blend, add it to your own memo
z1 = h @ W1 + b1; a1 = np.maximum(0, z1) # workers, then the manager
lg = a1 @ W2 + b2 # one score per word
p = np.exp(lg - lg.max()); p /= p.sum()
return x, K, Vv, q, s, w, h, z1, a1, p
def show(sentence):
ids = [stoi[w] for w in sentence.split()]
*_, w, h, z1, a1, p = forward(ids)
top = p.argsort()[::-1][:2]
print(f" {sentence:34} -> " + ", ".join(f"{words[t]} {p[t]*100:.0f}%" for t in top))
print(" where 'it' looked: " + " ".join(f"{t}:{wi:.2f}" for t, wi in zip(sentence.split(), w)))
print("before training:"); show("the dog saw the cat and it")
lr = 0.3
for step in range(1001):
gE,gWQ,gWK,gWV,gW1,gb1,gW2,gb2 = [np.zeros_like(a) for a in (E,WQ,WK,WV,W1,b1,W2,b2)]
loss = 0
for ids, y in data:
x, K, Vv, q, s, w, h, z1, a1, p = forward(ids) # 1. FORWARD
loss += -np.log(p[y]) / len(data) # 2. LOSS
# 3. THE REPORT walks down: judges -> workers -> the reading room -> embedding rows
d = p.copy(); d[y] -= 1; d /= len(data)
gW2 += np.outer(a1, d); gb2 += d
d1 = (d @ W2.T) * (z1 > 0) # silent workers get zero
gW1 += np.outer(h, d1); gb1 += d1
dh = d1 @ W1.T
dx = np.zeros_like(x); dx[-1] += dh # the memo's own path
dV = np.outer(w, dh); dw = Vv @ dh # back through the blend
ds = w * (dw - w @ dw) / np.sqrt(D) # back through softmax + scaling
dK = np.outer(ds, q); dq = K.T @ ds # back through the scores
gWK += x.T @ dK; dx += dK @ WK.T
gWV += x.T @ dV; dx += dV @ WV.T
gWQ += np.outer(x[-1], dq); dx[-1] += dq @ WQ.T
for t, i in enumerate(ids): gE[i] += dx[t] # only rows that appeared
# 4. EVERYONE ADJUSTS: the three stamps, the table, the workers, the judges
E -= lr*gE; WQ -= lr*gWQ; WK -= lr*gWK; WV -= lr*gWV
W1 -= lr*gW1; b1 -= lr*gb1; W2 -= lr*gW2; b2 -= lr*gb2
if step in (0, 25, 50, 100, 200, 500, 1000): print(f" step {step:4d} loss {loss:.3f}")
print("after training:")
for s in ["the cat moved and it", "today the dog moved and it", "the cat saw the dog and it",
"the dog saw the cat and it", "today the bird saw the cat and it"]:
show(s)
Three things to notice before running it.
forward has the entire reading room in five lines: stamp the labels and contents, write the slip, score, share, blend-and-add. Everything from the six-step walkthrough, in order.
E, WQ, WK and WV are all in the update line at the bottom, next to the workers and the judges. The three stamps are weights, and the report reaches them.
q = x[-1] @ WQ is one row of Q, not the whole grid. The last memo is the only reader whose reading anyone downstream uses, so it's the only one the toy computes.
What you get
before training:
the dog saw the cat and it -> it 65%, the 9%
where 'it' looked: the:0.14 dog:0.15 saw:0.06 the:0.13 cat:0.24 and:0.20 it:0.07
step 0 loss 3.928
step 25 loss 0.418
step 50 loss 0.022
step 100 loss 0.003
step 200 loss 0.001
step 500 loss 0.000
step 1000 loss 0.000
after training:
the cat moved and it -> purred 100%, barked 0%
where 'it' looked: the:0.01 cat:0.95 moved:0.03 and:0.02 it:0.00
today the dog moved and it -> barked 100%, chirped 0%
where 'it' looked: today:0.00 the:0.01 dog:0.99 moved:0.00 and:0.00 it:0.00
the cat saw the dog and it -> purred 100%, barked 0%
where 'it' looked: the:0.00 cat:0.96 saw:0.01 the:0.00 dog:0.03 and:0.00 it:0.00
the dog saw the cat and it -> barked 100%, chirped 0%
where 'it' looked: the:0.00 dog:0.99 saw:0.01 the:0.00 cat:0.01 and:0.00 it:0.00
today the bird saw the cat and it -> chirped 100%, barked 0%
where 'it' looked: today:0.00 the:0.00 bird:1.00 saw:0.00 the:0.00 cat:0.00 and:0.00 it:0.00
The predictions are right, but as with the first model, they aren't the interesting part. Read the where 'it' looked lines.
Before training the shares are a random smear, 0.06 to 0.24, with cat slightly ahead by luck. After a thousand rounds the same room gives 0.99 to dog. Across five sentences, with the subject in position 1 or 2, with or without a distractor, the share on the subject is between 0.95 and 1.00. That is a room that learned to find the subject of a sentence, and the only instruction it was ever given was "predict the next word, here's how wrong you were."
One detail worth a sentence: the starting loss is 3.93, and pure guessing among twelve words would be ln(12) ≈ 2.48. The untrained model is worse than guessing, because random judges aren't neutral. Before training it says it with 65% confidence. Random weights have opinions; they're just wrong ones.
Three experiments
Experiment 1: delete the stamp. Change one line:
x = E[ids] # was: E[ids] + PE[:len(ids)]
Run it:
step 500 loss 0.467
step 1000 loss 0.463
the cat moved and it -> purred 100%, barked 0%
the cat saw the dog and it -> purred 54%, barked 46%
the dog saw the cat and it -> purred 54%, barked 46%
The single-animal sentences are still perfect. The twins get the identical answer, wrong for one of them. Without stamps, the room receives the same set of labels and contents for both sentences, just in a different order, and the reading room is order-blind by construction: sum of share-times-contents doesn't care which came first. This is permutation invariance again, and it's exactly why Desk 3 exists.
And the 0.463 isn't arbitrary. Six sentences perfect, twelve at 50/50, averaged: 12 × ln(2) / 18 = 0.462. The loss is telling you exactly what the model can't do.
Experiment 2: give everyone an equal share. Two lines: the share line, and the line that sends the report back through it (there's nothing to correct any more).
w = np.ones(len(ids)) / len(ids) # was: softmax
ds = np.zeros_like(w) # was: back through softmax
Loss stalls at 0.467, and the twins both come out barked 54%, purred 46%. That's Idea 2 from the top of the article, failing on cue. Average-and-hope can't pair a word with its position; only a slip scored against labels can.
Experiment 3: change the seed. np.random.seed(5) → 4. Predictions still perfect. Now look at one of the rows:
the dog saw the cat and it -> barked 100%, purred 0%
where 'it' looked: the:0.03 dog:0.06 saw:0.20 the:0.09 cat:0.60 and:0.00 it:0.01
Right answer, and the room is staring at cat. With eighteen sentences there's more than one arrangement that works, and this run found one where the workers decode "the object is cat, so the subject isn't" from a cat-heavy blend. The trained room isn't obliged to be readable. Worth remembering the next time a real model's attention heatmap looks like noise: it might be, or it might be a solution that just doesn't look like one.
Generating text, and why the first token is slow
You watched the loop run on the first model: judges vote, the winner is glued on, the building runs again. A real model runs the identical loop; the only change is that the memo now reads the whole context in the reading room instead of a two-word window. Here's what that costs in the reading room.
step 1 the dog saw the cat and it → barked
step 2 the dog saw the cat and it barked → and
step 3 the dog saw the cat and it barked and → ran
At step 2 the new memo is barked. What does it need? Its own request slip, written fresh. And the labels and contents of every earlier memo, to score against and blend from.
Now the question that matters: did any of those earlier labels and contents change between step 1 and step 2? No. dog's label is dog's vector through a frozen stamp. Same vector, same stamp, same label. And because of the causal mask, dog never read barked, so nothing that arrived later can alter what dog already wrote. The only new work at each step is the newest memo's slip, label and contents.
So don't recompute them. Store them. That's the KV cache: the labels (K) and contents (V) of every memo so far, kept in memory, added to once per token.
In the toy, K and Vv are the cache and q is the fresh part. The room recomputes everything on every call because the toy is tiny; a real model never does.
Two phases, two speeds
This is why a chatbot pauses before its first word and then streams the rest.
Prefill. Your whole prompt arrives at once. Every memo in it needs a label and contents on every floor, and none of them are cached yet. For a long prompt that's a large amount of arithmetic, all of it before the first token can be chosen. That's time to first token.
Decode. From then on, each step stamps one new memo and reads the cache. That's tokens per second.
They're different numbers on every benchmark, and now you know why. A model can have a slow prefill and a fast decode, or the reverse, depending on hardware.
Why long context is hard
The cache grows with every token, on every floor. In the simplest design, one label plus one contents per memo per floor is 2 × 768 numbers. Twelve floors: 18,432 numbers per token. A 100,000-token context is 1.8 billion numbers sitting in GPU memory for the length of your conversation, roughly 3.7 GB at 16-bit precision, before you count the model itself. Double the context, double the cache. (Production systems shrink it: sharing labels across rooms, storing them at lower precision. Different article.)
That memory is why long context is priced the way it is, and why "128K" is a harder engineering promise than it sounds.
One room isn't enough
The toy's room learned one relationship: pronoun → subject. A real sentence has many at once. Which noun does the verb belong to? Which word came just before this one? Which earlier sentence is this one about? Is this bank near river or near loan?
One room has one set of three stamps, so one kind of slip, so one kind of lookup. Ask it to find subjects and adjacent words and topics with a single request slip and the questions fight over the same eight numbers.
The fix is the obvious one. Several reading rooms on each floor, side by side, each with its own three stamps. Every memo goes through all of them at once and comes out with several blends. Formally, multi-head attention; each room is a head.
The budget
The pipe through the building is 768 wide. Rather than give each of eight rooms its own 768-wide slips, slice the pipe: each room's slips, labels and contents are 96 numbers wide, 768 ÷ 8. Each room's blend is 96 wide too. Lay the eight blends end to end and you're back at 768, then one more stamp, W_O, mixes them into the memo that goes on to the workers. That's the fourth trainable stamp mentioned earlier.
The original transformer used 8 rooms of 64 on a 512-wide pipe. GPT-2-small uses 12 rooms of 64 on 768. Twelve rooms on each of twelve floors is 144 reading rooms in a model you can run on a laptop.
Why 8 or 12, and not 80? It's a fixed budget. More rooms means narrower slices: 768 ÷ 80 is under ten numbers per slip, and ten numbers is too few to encode "animal, early, and not the one being looked at" alongside everything else a label has to advertise. Too few rooms and every relationship in the language fights over one slip. It's the same kind of compromise as the 768 at Desk 2: settled by trying things, not derived.
And, as with everything else in this series, nobody assigns the rooms their jobs. They start with random stamps and specialise the way the workers did in Part 1, each one corrected only by the mistakes it had a hand in.
The whole pipeline
raw text
↓ tokenize dictionary frozen
↓ embed one row of 768 per token weights
↓ add position stamp formula
↓ floor 1
reading rooms 3 stamps per room, plus W_O weights ← new
workers then the manager weights
↓ floors 2 … N same again
↓ judges one score per word weights
Each token leaves the lobby knowing what it is and where it sits, and leaves every floor knowing more about what's around it. That's the whole journey from text to attention. By the top, bank has read river a dozen times over, and it has long since found its noun.
What's missing is the housekeeping that lets you stack a hundred of these floors without the influence report fading to nothing on the way down, and the last step from the judges' scores to actual text. That's the next article.
Recap
Tokenization cuts text into subword pieces using merges learned from frequency. Common words stay whole, rare ones split, nothing is unreadable. The dictionary it produces is frozen before training and never changes.
Embeddings turn each token ID into a row of 768 numbers. Those numbers are weights: they start random and are shaped by the training loop like everything else. Similar tokens get similar rows because they appear in similar contexts, and that's the only reason.
Position stamps are added to each embedding so the same word at different positions produces different vectors. Without them, the model sees a set of words rather than a sequence, and the reading room is provably order-blind.
The judges are the dictionary lined up: one per word, each reading the last memo's tray with their own weights and writing one score. Highest wins, the word is glued on, the building runs again. The model returns scores; the loop around it chooses, remembers, and stops.
Attention is a weighted average of the other tokens' vectors, where the weights are computed from the tokens themselves, per sentence, rather than stored in the file. Three stamps turn every memo into a request slip (query), a folder label (key) and contents (value). Scores are slip-against-label dot products, scaled by √dₖ, turned into shares by softmax. The blend of contents, weighted by share, is added to the memo.
The stamps are weights. The shares are not. Backprop reaches the stamps through the shares; the shares themselves are recomputed every sentence and thrown away.
The cache exists because earlier memos' labels and contents never change. Prefill fills it, decode reads it, and its size is why long context is expensive. Heads are several rooms per floor, each with its own stamps, splitting a fixed budget.
The training loop is unchanged from Part 1. The only additions are a lookup table at the bottom, which is also trained, four stamps per reading room, and a row of judges at the top instead of a single output.
The vocabulary
| Building | Formal term |
|---|---|
| Desk 1, the dictionary | Tokenizer (BPE) |
| Desk 2, a row of 768 numbers | Embedding |
| Desk 3, the position stamp | Positional encoding (sinusoidal, or RoPE) |
| A memo | A token's vector, the hidden state |
| A tray | One layer's activations for one token |
| The judges, one per word | The unembedding / LM head |
| The judges' scores | Logits |
| Highest wins, or roll the die | Greedy decoding, or sampling with temperature |
| Glue it on, run again | Autoregressive generation |
| The reading room | Self-attention layer |
| Request slip | Query, q |
| Folder label | Key, k |
| Contents | Value, v |
| The three stamps | W_Q, W_K, W_V |
| Slip against label | Dot product, q · k |
| Divide by √8 | Scaling by √dₖ |
| Shares that add to 1 | Attention weights (softmax) |
| The blend | Attention output |
| Add it to your own memo | Residual connection |
| A memo can't read ahead | Causal mask |
| Stored labels and contents | KV cache |
| Several rooms per floor | Multi-head attention |
| The mixing stamp | W_O, output projection |
Six things to remember
The embedding table is a third of the model, and it's trained. Backprop doesn't stop at the first layer. It reaches into the lookup rows.
Meaning comes from context, not definitions. Two words end up with similar vectors because they appear in similar sentences. Nothing else is optimised for.
Everything at the three desks is provisional. A token leaves the lobby with its isolated meaning and its position, and no idea what's around it. The reading room is where context gets added, one floor at a time.
The weights that matter in attention aren't in the file. A worker's points are fixed at the freeze; a request slip is written per sentence by another token. Attention weights are activations. The stamps are the parameters.
Nothing was told to find the subject. The slip learned "is it an animal, and is it early" from next-word prediction alone, the same way
kingandqueendrifted together and the same way the both-detector appeared in Part 1. Rooms in real models learn pronoun-to-noun, verb-to-subject and previous-word the same way, and nobody labels them.The top floor is the dictionary, lined up. One judge per word, each holding a scorecard for the last memo. The model's entire output is that row of scores. Everything else you experience as "the model talking" is a ten-line loop: read the scores, pick, glue on, run again.
Run both models. Watch 0.577 climb to 0.988, and dog:0.15 become dog:0.99. The first is a model discovering that two words mean the same kind of thing. The second is a model discovering what "it" refers to. Neither was told, and both were learned from nothing but which words came next.
