Published · 11h agoScience2 min read
The number in circulation says one prompt sentence flipped a model ranking and explained 77% of label variation. The supplied text of the Oracle Problem essay stops before any of that appears.
Written for builders.See today for builders
Compiled by The ScientistSomething wrong?How this is made
"Explained label variation" would have to be a variance decomposition: fit the labels against a set of factors, set aside the part nothing accounts for, then report how much of the remainder one factor owns. That construction hides its own denominator. A prompt sentence can own most of the explained share while the explained share itself is thin, and the two readings imply opposite things about how much eval wording costs a buyer. Nothing in the supplied text sets out the label set, the models compared, or the two prompt variants, so there is no way to tell which reading applies [10].
The awkward part is that the essay predicts this behaviour anyway. Its authors argue there is no shared, absolute definition of ground truth in medicine, and that any eval built on clinical data ends up scored against what one physician did on one particular day [4]. If that holds, a benchmark number is partly a property of the sentence used to elicit the label, which is close to what a16z and Protege say outright: a good benchmark result is as much a statement about how you wrote the test as about clinical merit against a different model or a different evaluation [3]. A ranking that reverses on a rewrite is the described system working as described.
What the text does put on the record is smaller and checkable. The sampling was millions of records drawn from nearly a trillion tokens of EMR text across billions of notes, labelled descriptive rather than causal [5]. Part way through 2026, about 1,686 notes per million carried a patient asking for clarification on something a model had said [6], roughly one note in 593 [11]. That sentence is where both copies run out [8][9]. The remedy the authors endorse is the slow one, following the patient forward long enough to see who was right [13], with Protege positioned to do it [14].
Ranked by verification strength, evidence, and original report placement.
The a16z version of "The Oracle Problem: an invisible bottleneck to AI and medicine" states the post was done in collaboration with Protege, and points to a podcast with Engy Ziedan, Protege's Chief Scientific Officer.
The version on withprotege.substack.com states the piece was originally published in a16z's newsletter.
The authors write that "The model did well on the benchmark" is as much a statement about how you wrote the test as a statement about clinical merit versus a different model or a different evaluation, and call this a kind of principal-agent problem.
The authors write that any benchmark or eval designed on clinical data is quietly opinionated, scored against what the physician did that particular day, and that there is no shared, absolute definition of "ground truth" in medicine.
The figures come from Protege's partner network: millions of records randomly sampled from nearly a trillion tokens of EMR text across billions of notes, and the authors describe the figures as descriptive rather than causal estimates.
Part way through 2026, about 1,686 per million notes recorded interactions where the patient wanted clarification on something a model said; explicit references to patient AI use were essentially absent through 2023 before rising sharply after GPT launch.
The reporting this story was synthesized from, earliest first. Every link goes to the original.