Two different models top the ATLAS long-context benchmark at 128K and at 1M tokens

ATLAS scores each model across a predefined grid of context lengths, from 8K to 1M tokens, instead of at one prompt size ^[5]. That design is how two models can each hold a lead. Gemini 3.1 Pro Preview led at 128K ^[3] and Claude Opus 4.6 led at 1M ^[4]. The two lengths are about eight times apart ^[14], so the lead changed hands somewhere between them ^[15]. The post does not report the scores, the margins, the uncertainty intervals or where the crossover falls.

The paper's stated motive is two weaknesses in older evaluations. Performance can deteriorate as input length increases, and success on foundational tasks such as retrieval does not necessarily transfer to complex application workloads ^[12]. ATLAS therefore separates foundational operations from downstream workloads ^[6]. The post puts the old question this way: the issue is no longer whether a model accepts a million tokens but how much of that context it can use correctly ^[13]. Acceptance is the easy part. The API takes the tokens and answers confidently either way, according to the post.

The aggregate is built for general ranking. ATLAScore is designed to penalize imbalanced capability profiles ^[7], and the framework propagates uncertainty into the final score ^[8]. Both are sound choices. The second lets a reader ask whether a gap between two models is larger than noise.

For either leader to transfer to your system, your prompts have to land near that length and your task mix has to resemble the eight ATLAS capability dimensions ^[2]. The lead also has to exceed the noise.

The post itself warns against the broad reading. It says the result does not make Claude universally better than Gemini, and that it means only that the relative ranking changes under the evaluation conditions ATLAS used ^[10]. We agree. In our view the useful test is a replay of a sample of your real prompts, at your real lengths, against two or three candidates.

This is a secondhand read. The post says it analyzes published research, is not an independent benchmark, and that its author did not personally test all 26 models ^[11]. The paper is arXiv 2605.28079, by Deli Huang and colleagues, published May 27, 2026 ^[1].