Anthropic puts three evaluator access tiers in writing, with METR at the top

The size of the publication right depends on a list Tech Times did not report: the pre-defined categories Anthropic may require evaluators to redact ^[4]. METR can call out a redaction it believes is unjustified, provided it does not reveal the redacted content, and Tech Times describes the right as unconditional because Anthropic cannot revoke it, per the published framework ^[16]. "Cannot be revoked" is the accurate claim, or rather the narrower one: the right survives, and so do the redactions.

In our view the tiers sort evaluators by how much they are allowed to say, and the older position survives at the bottom. Tech Times says the post is the first time Anthropic has put in writing what evaluators at each level may see, access and publish ^[12]. Tier 1 keeps Anthropic's approval over publication ^[3]. Tier 1 approval is the nearest thing in the framework to the strict editorial authority Reuters reported Anthropic had sought ^[15].

The framework itself is a post on Anthropic's website ^[2], although METR's engagement runs through a small team under enforceable agreements ^[7]. The counter-thesis is that a public dispute over a redaction costs a lab more than the redaction saves it, and six embedded researchers with a legal right to say so make that cost real ^[1]. We would be wrong about the tiers if the redaction list turns out to be short and public.

The money in the account sits with a different evaluator. In September Anthropic named Faculty its first embedded evaluator, and the two sides each pledged at least $1 billion over five years ^[10], or at least $200 million a year each ^[14]. METR sits at the top tier and the two other organizations named sit below it ^[13]. The only dollar figure attached to any of the three belongs to Faculty, which has not reached Tier 3 ^[11].