Goodfire researcher argues InceptionV1's incoherent last layer came from architectural problems and weak interpretability tools, not superposition
Goodfire researcher says 99-100% of InceptionV1's last-layer neurons are easily interpretable after architecture changes. The essay argues the superposition evidence was misread, so networks can be studied directly.
The Scientist · Science desk
What happened
- At first study, almost none of the neurons in InceptionV1's last layer were easily interpretable, and the essay says it was unclear how to study the layer at all.
- The author blames architectural problems that made later layers overflow, which weak interpretability tools then amplified into an appearance of incoherence.
- Versions of InceptionV1 trained across a 300x range in size stay broadly legible, and the smaller ones get coarser neurons, such as one car neuron for many kinds of car.
- The 2019 circuit that first illustrated superposition showed a car feature spreading into unrelated dog neurons, and the author found it with Chris Olah.
Why it matters
- Who has to decide decision Groups that extract features with a reconstruction objective have a reason to test, model by model, whether the original neurons can be read directly first.
- What it costs cost The unfolding route costs a second trained model, and the study then describes the approximation instead of the network.
- What it limits constraint The legible InceptionV1 is a retrained variant with architecture changes, so the result applies to networks built that way and says nothing yet about models already deployed.
The InceptionV1 comparison is a tidy design. Model size and training data stayed fixed while the architecture changed [8]. If a shortage of neurons had left the last layer unreadable, a network of the same size on the same data should have stayed unreadable, and the author reports it did not [8].
The size sweep is the sharper test, because the two accounts disagree about what a small network does. Under superposition, a network that runs short of neurons folds unrelated things together [3]. The author reports that the smaller versions get coarser neurons instead [9].
The 2019 circuit carries the other half of the argument. The author wrote: "It wasn't hiding anything; it was ordinary generalization and composition." [13] The author's name for the replacement account is recursive generalization [6]. By the author's own telling, the canonical vision evidence for superposition rested on one example: that circuit was the only one shown and the only one the author ever saw [11]. The author also says nothing reads the car-and-dog combination [12].
The thing this doesn't tell you is whether the networks people deploy today are clean. The legible InceptionV1 is a version with changed architecture [8], so the result describes networks built that way. The 99-100% figure is the author's own count. The passages we have stop partway through the vision examples [5], before any language-model results, and do not say how "easily interpretable" was judged.
The essay's summary is "Neural networks are already clean." [14] We think the InceptionV1 result earns that sentence for InceptionV1 with its fixes, and not yet beyond it. For the language models the author says they studied [1], it is a hypothesis with a plain test: read the neurons directly, with no second model, and count.
What to watch
- A second published circuit in which a feature spreads into unrelated neurons that downstream neurons do read, which would restore the superposition reading.
- Independent reproduction of the 99-100% last-layer count on InceptionV1, with the criterion for 'easily interpretable' stated.
- Results on language models showing whether neurons can be read directly there, without a sparse autoencoder.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence30
- Adoption
- Not scored
- Hype gap+35
- Incentives
- Not scored
- Confidence40
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The essay is published on Goodfire's site, is written in the first person, and says its author spent many months studying neurons and circuits in image and language models before reaching its conclusion.
- [2]
The essay describes two approaches to mechanistic interpretability: studying a network's neurons and circuits directly, or starting from a list of hypothesized features and extracting them with increasingly sophisticated reconstruction objectives, usually ignoring the original network.
- [3]
Under the superposition explanation, a network that runs out of neurons folds unrelated things together because they do not interfere, and studying it means training a second model, such as a sparse autoencoder, to approximate the network as it would look unfolded and studying that instead.
- [4]
The last layer of InceptionV1, the early field's model organism, seemed so full of arbitrary mixtures that it was unclear how to study it: almost none of its neurons were easily interpretable.
- [5]
The essay text available breaks off partway through its discussion of the vision examples.
- [6]
The author says the canonical evidence of polysemanticity and superposition, which convinced much of the field including the author, was misread, and that what looked like arbitrary packing was mostly a motif the author calls recursive generalization.
- [7]
The author says superposition was the wrong explanation for InceptionV1: architectural problems made later layers overflow, and weak interpretability tools amplified these problems, making the late layers seem incoherent.
- [8]
The author reports that with architectural changes designed to help deep layers learn better, and no change to model size or training data, 99-100% of neurons in InceptionV1's last layer are easily interpretable.
- [9]
The author trained versions of InceptionV1 across a 300x range in size and says it is broadly legible at every scale; with less room, neurons cover coarser-grained concepts, such as one car neuron covering many kinds of car, and with more room that neuron differentiates into many specific cars.
- [10]
In 2019 the author and Chris Olah found the circuit originally introduced as the superposition phenomenon, in which a car feature appeared to spread into a set of unrelated dog neurons.
- [11]
The author says the car-into-dog-neurons circuit was the only one they showed and the only one they ever saw.
- [12]
The author says nothing reads the combination of car and dog neurons in that circuit.
- [13]
"It wasn't hiding anything; it was ordinary generalization and composition."
- [14]
"Neural networks are already clean."
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- important essay
already-clean.goodfire.com
1 article · October 15, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
- SuperpositionFollow
- Mechanistic interpretabilityFollow
- Sparse autoencodersFollow