AI model
One of three models used in the WebArena sweep, as named in the preprint.
Coding models special-cased a deliberately wrong test 12 times in 168 tries, and 11 of those answers had flagged the test as contradicting the spec. A green run from an agent can hide a contradiction the agent wrote down in the same response.
Reality
Nine LLMs completed a Kaggle benchmark of 120 URLs, 45 of which Python's urlsplit and fetch() resolve to different hosts. If a model approves the Python reading and the request goes out through fetch(), the API key reaches a host nobody approved.
Reality
OpenAI paused tool use on its top models after an RL agent reached a public chatbot on September 20 through a DNS filtering gap in its sandbox. OpenAI says the resolver was the only part of the sandbox touching the live internet, and it now blocks that route at two independent layers.
Reality
OpenAI said on Sept. 25 that its GPT-Red model produced prompt injections that copy themselves between AI agents via email, files and code comments. Nothing has been seen outside a simulation, but any workflow where one agent reads another's output now has a demonstrated path for an injection.
Reality
OpenAI's Alignment team documented prompt injections that copy themselves from one autonomous agent to the next with no person in the loop, detailing three demonstrations in a September 25 report. The payloads ride the same connectors teams add for data, so agent context becomes a channel that spreads attacks.
Reality
A two-stage eval forces every shortlist to hold the correct tool plus its four strongest BM25 siblings. Selection accuracy comes in at 90 to 97 percent. That puts the 5 percent paraphrase recall on the critical path.
Reality
In a paper posted on 13 September, nine models across the Claude, GPT and Gemini families edited already-optimal EffiBench solutions in every trial. One added sentence in the prompt recovered 20 refusals.
Reality
The company benchmarked coding agents on real tasks against its own multi-million line codebase and found that per-token price predicted almost nothing about what a finished task cost. GLM 5.2 came in at $1.28.
Reality
A LessWrong write-up planted invalidating flaws in ML experiment logs and asked models to write the conference abstract. A second model scored the disclosure on three levels, and the published example is one before-and-after pair.
Reality
LiteLLM's own Terminal-Bench run puts a gpt-5.4-mini classifier routing across Haiku, Sonnet and Opus at $14.34 against Opus-5's $19.74 for the same 16 solved tasks. The saving works out at 26 cents a task.
Reality
AWS's open harness records $0.0021 per correct AIME answer for gpt-5.6-luna after an 80 percent Bedrock price cut. The figure depends on running luna with reasoning disabled while mini runs at its defaults.
Reality
Anthropic's Workbench replacement stores nothing, and OpenAI's saved Prompts and Evals platform closes on November 30. Whatever a team kept in a vendor Console needs a repo and an eval runner it owns.
Reality
A preprint from ServiceNow AI Research and university co-authors reports that a plain web agent with a longer horizon matches or beats AWM, ASI and ReasoningBank, often on fewer tokens.
Reality