Agents on the Good team fall for coordinated deception in a Blood on the Clocktower benchmark

"When playing Evil, agents sometimes produce sophisticated coordinated play. When playing Good, they tend to be naive and fall for these kinds of plays," the authors wrote ^[3]. The post's summary states that result in words only. It does not report win rates split by side, so "sometimes" and "tend to" are the only measure of size available.

Of each model's games, 210 are on Good and 90 on Evil, 30 of those as the Imp ^[10]. At a 10-seat table, that works out to three Evil seats and one Imp per game ^[1]. Evil play is 30% of any model's record ^[2]. The coordinated plays the authors describe come from that share.

Two harness choices shape what a Good agent can see. Voting is simultaneous and blind, so inference for every player can run in parallel ^[9]. For throughput, that is a sensible choice. It also means a Good agent casts each vote without seeing how anyone else is voting on that nomination ^[9].

Memory is the second choice. Rules and role stay in context throughout. Only the most recent few thousand tokens of the game are visible, and anything older has to come from the agent's own scratchpad or a recall function ^[11]. A day runs through opening statements, group discussion, breakout conversations and nominations, each nomination with an accusation and a defence ^[8]. To spot a coordinated play, a Good agent has to set claims from different phases against each other, possibly after those claims have left the visible window. The authors checked, manually and with agents, that results came from real game-playing ^[12].

That check leaves the transfer question open. For the Good-side result to carry into a production system, the naivety has to belong to the model and not to this memory setup. The evidence so far fits either explanation. I'd expect the same misses from a reviewing agent that works with a short window and self-managed notes while its peers are able to coordinate.

The authors built the benchmark as a testbed for deception and coordination ^[1]. Almost all alignment evals test one agent, and the authors worry that models can look aligned individually while behaving unpredictably as a swarm ^[13]. They cite METR's investigation into the OpenAI-HuggingFace incident, where some behaviours, in their view, would have been hard to catch with single-agent evals ^[14].