Security of AI Evaluation Organizations

Attackers talked a METR researcher's agent out of its inference API key

METR's inference key sat on a researcher's personal EC2 instance, and the agent handed it over when asked. The three-week burn would have cost about $600,000 if the model provider had not donated the credits.