Model-as-judge evaluation limits

Anthropic's Insights pilot sets the ceiling on what outside evaluators can verify

Three groups received counts and cluster descriptions from roughly 250,000 conversations each. The raw chats stayed on Anthropic's servers, and so did the classifier that produced the numbers.