The ARC-AGI Premise Is A Working Test For Real Model Gains, And It Closes 8 November

The 2026 ARC-AGI prize launched in March and closes to submissions on 8 November, according to Forbes contributor John Werner ^[1][8]. What makes it worth an operator's attention is not the leaderboard but the scoring rule underneath it, which offers a practical way to tell a genuine capability gain from a model that has simply absorbed more of the test.

The premise comes from Francois Chollet's book On the Measure of Intelligence, quoted on the ARC-AGI site: intelligence is "a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty" ^[2]. The operative words are efficiency and acquisition. A system that already holds the answer is not demonstrating intelligence under this definition; a system that works out a novel answer quickly, from little, is.

ARC-AGI builds on that by scoring fluid intelligence, the ability to reason, solve novel problems, and adapt to new situations, rather than crystallized intelligence, which rests on accumulated knowledge and skills ^[3]. The team's stated reason is fairness: crystallized intelligence includes cultural knowledge and learned information that would give a machine an unfair advantage in a comparison with a human ^[4]. To pin down the starting priors every solver is allowed, the design leans on Elizabeth Spelke's core knowledge theory, the cognitive building blocks present at birth or acquired very early with minimal instruction ^[5]. Those systems cover objects, number, agents, geometry, and social partners ^[6].

There is a lesson in one detail. The ARC-AGI writers note that a benchmark built around tasks in written English "would immediately disadvantage any AI" ^[7]. That is the crystallized-knowledge trap in miniature: a test that rewards what a system has taken in rather than what it can work out. It is also how most published benchmark gains read once a test set has been in circulation long enough to leak into training data.

For a leader deciding whether a new model release is progress or marketing, the distinction is usable without a neuroscience degree. Ask what changed. If a score rose because the model saw more examples of that task, the gain is crystallized and will not travel to the novel problem sitting in your own operation. If it rose because the model needed fewer examples to reach the same skill, that is the efficiency Chollet is measuring, and it is the kind that carries over to work you have not scripted.

Watch the 8 November deadline, then watch how far the winning designs move ARC-AGI's public numbers ^[1]. Watch too for vendors quoting benchmark jumps without saying whether the benchmark was novel to the model. Under this framework, that omission is the whole question.