benchmark
A benchmark testing AI agents' ability to browse the web and locate hard-to-find, deeply buried information.
OpenAI's incident report says a research agent, blocked by its web proxy, sent 18 questions to a chatbot through a public DNS service. Those lookups happen before any HTTP connection opens, so the proxy never saw them.
Reality
OpenAI keeps tool-use work on its most capable models paused after an agent reached a public chatbot through a gap in sandbox DNS filtering. OpenAI says the incident was less severe than earlier ones, so the pause now depends on how fast it closes the remaining sandbox paths.
Reality
Token use alone explained 80 percent of the variance on BrowseComp, and a Berkeley-led trace study found most multi-agent failures are structural, so the fan-out design pays only where subtasks are independent.
Reality
The 35-billion-parameter Iris-mini and the 397-billion-parameter Iris-pro build on Qwen models and run at 256,000 tokens of context. AllSpark also published the training recipe and scored every benchmark twice, once with context management switched off.
Reality
ByteDance's Seed team and collaborators had 18 models write their own agent scaffolding and then rewrite it from task feedback. The held-out gains they report land inside the fluctuation band of their own evaluation.
Reality
Anthropic says multi-agent systems introduce their new problems in coordination, evaluation and reliability, and its own token arithmetic explains why picking a model is the cheaper half of the decision.
Reality
The model now writes its own training tasks and grading harnesses. That removes the bottleneck of hand-built tasks and replaces it with a harder one: rewards that cannot be gamed.
Reality