More Capacity, Worse Fit
In 1986 Rumelhart, Hinton and Williams published the backpropagation paper, and tucked inside it is a small experiment that gets cited far more than it gets run. They built two family trees, one English and one Italian, structurally identical, and trained a network to answer questions of the form "person, relation, who?" The network was given no hint that the two trees were separate, no hint that people have generations, no hint of anything except individual pairs. Then they looked at what the hidden units had settled on, and the units had settled on generation, nationality, and which branch of the tree you sat in.
That is the result everyone remembers. Structure nobody specified, falling out of gradient descent.
I rebuilt it in numpy, by hand, because reading a backprop derivation and writing one are different activities. No autograd, no framework. The gradient check against finite differences agrees to 3e-9, which is the only part of this I am completely sure about.
The setup
Two trees, 24 people, 12 relations: father, mother, husband, wife, son, daughter, brother, sister, uncle, aunt, nephew, niece. Every pair that has at least one answer becomes a training row, which gives 132 rows. Answers can be plural, so David, son wants both James and Robert, and the target is multi-hot.
The architecture matters more than it looks:
person (24) --> person encoder (24 -> N) --+
+--> hidden (24) --> output (24)
relation (12) --> relation encoder (12 -> 6) --+
The two encoders are kept apart on purpose. The person encoder never sees which relation is being asked about, so whatever those N units hold has to describe the person and nothing else. If the relation could leak in, calling the result a "person representation" would be a lie.
Then you train it twice. N=24 gives one unit per person, which is enough room to assign everyone their own coordinate and memorise. N=6 puts four people per unit, so the codes have to overlap and something has to be shared.
The expectation going in is obvious. The wide one fits easily, the narrow one struggles a bit on the training set but generalises better, and that tradeoff is the whole lesson.
What actually happened
Ten seeds, four held-out rows each:
| person_dim | train | test | final loss |
|---|---|---|---|
| 24 | 0.988 | 0.425 | 0.1982 |
| 6 | 0.997 | 0.575 | 0.0372 |
The narrow model fit the training set better. Not the test set, the training set. At about a fifth of the loss. The wide model failed to reach a perfect training fit in 6 of 10 seeds, and it was still slowly improving at 20,000 epochs, while the narrow one was done by about 4,000.
This is not the tradeoff story. The tradeoff story is about capacity, and capacity was never the binding constraint here. The 24-unit model contains the 6-unit model as a special case. Anything the small one found, the big one could represent. It just did not find it.
So the interesting claim is not about generalisation at all. It is about the loss surface.
Where the computation lives
Here is the argument I find convincing, and it follows from the architecture rather than from speculation.
The person encoder takes a one-hot vector, so it is an embedding table with a sigmoid on top. Row i, squashed. That is all it is.
With N=24, that table can hand the hidden layer 24 near-orthogonal codes. It can preserve identity and compress nothing. Which means the hidden layer, all 24 units of it, receives something close to "person 7" with no information about how person 7 resembles person 13, and has to learn all 132 rows as effectively separate cases.
With N=6 the codes have to collide. Two people who behave the same way under many relations can share coordinates, because there is no room not to. The encoder is forced to do work, and the work it does makes the hidden layer's job easier. Similar inputs already arrive looking similar.
So the bottleneck is not only a constraint. It is an inductive bias that moves computation into a layer where it is cheap, and the payoff shows up during optimisation, before generalisation ever enters the picture.
I want to be careful here, because that is an explanation and not a measurement. Three things would falsify or sharpen it, and I have only done the third:
- Check whether the wide encoder is saturating. Track the mean magnitude of the sigmoid derivative in the encoder over training. If it collapses early, the stall is saturation and not geometry.
- Widen the hidden layer for the 24-unit model. If the stall clears, the real bottleneck was downstream all along and my story is wrong.
- Rule out a plain hyperparameter artifact. Learning rates above about 1.0 diverge with momentum 0.9, and the wide model is still improving at 20,000 epochs, so it is slow rather than stuck at a floor. That much I did check.
The famous figure is a sample
Now the part the paper is known for. Print the six numbers for each person, scaled 0 to 9:
John 1 6 9 2 9 0 Giovanni 8 7 9 4 9 0
Mary 0 7 9 1 9 0 Maria 8 8 9 3 9 0
David 1 9 0 0 8 0 Marco 9 9 3 0 6 2
Linda 0 0 0 8 0 7 Francesca 9 2 1 8 0 8
Sarah 1 8 4 9 8 0 Giulia 7 9 6 9 8 2
Michael 0 8 6 8 9 8 Luca 5 9 7 9 8 9
James 0 0 0 5 0 9 Matteo 8 9 0 9 0 6
Emma 0 4 0 6 0 9 Sofia 9 5 0 8 0 9
Robert 1 7 0 8 0 6 Andrea 9 0 0 8 0 9
Alice 0 0 1 0 0 0 Chiara 9 0 3 3 2 4
Daniel 0 1 1 1 0 0 Paolo 8 9 4 7 3 6
Sophie 2 0 0 0 1 0 Elena 9 1 2 2 2 4
The first column separates the families. Every English person between 0 and 2, every Italian between 5 and 9. Nothing in the data says the two trees are separate. The network only ever saw individual pairs of people. But answers leaking across the trees is expensive, and one unit spending itself on nationality is the cheapest way to stop that, so that is what it does.
It is a genuinely nice thing to see with your own data.
It also happens in 6 runs out of 10. In the other four the distinction is still present but smeared across several units, with no single one you could point at in a figure.
I do not think that undermines the 1986 result. In 1986 the claim that needed making was that backprop can discover structure nobody put there, and one clean run is a perfectly good existence proof for that. But the sentence that gets passed down is usually that backprop does discover such structure, and those are different claims. Interpretable units here are something the training run sometimes produces, not something the architecture promises. Anyone who has stared at a feature visualisation and felt it click should probably know it took a few seeds to get there.
Worth noting too: the clean split appeared in 3 of 10 seeds when I only had eight relations, and 6 of 10 once uncle, aunt, nephew and niece were added. Those four reach further across the tree than parent and sibling do. More long-range structure in the data, more pressure to encode it in one place.
Defining the data was the hard part
The thing that took longest was not the network. It was deciding what "uncle" means.
The trees are built from three primitives: parents, spouse, sex. Everything else is derived. Father is a parent filtered to male, sister is a sibling filtered to female, and none of that needs a decision. Uncle does. Does your aunt's husband count?
Say no, and half the tree has no uncle at all, because the people who married in have no blood siblings of their own. Say yes, and you have committed to something on the other side too: his wife's sibling's children are now his nephews and nieces. The two choices are not independent. Once in-laws count for one, they have to count for the other, and the reward is that the relations become exact inverses:
for nibling in niblings_of(person): assert person in piblings_of(nibling)
for pibling in piblings_of(person): assert person in niblings_of(pibling)
That runs over all 24 people and it is the property that breaks first if either definition drifts. It is also the only test in the suite that checks the data rather than the code, which in retrospect is the one I should have written first. The ground truth in a supervised problem is not given to you. You choose it, and then you owe someone a reason.
Where this goes
The next thing I want to run is the hidden-layer experiment, because it is the one that could show my explanation is wrong. After that, plotting the encoder's derivative magnitude over training, which is cheap and would settle the saturation question in an afternoon.
Everything is numpy and it runs in under a minute on a laptop, which is the right size for a project you actually want to understand.
Paper: https://gwern.net/doc/ai/nn/1986-rumelhart-2.pdf
Code: https://github.com/VINODvoid/treeq