Skip to content

ScienceNot yet confirmed elsewhere1 publisher2 min readPublished

Explaining itself made the AI more persuasive and the reviewers less right

In a Harvard, MIT and University of Washington experiment, 228 reviewers deferred to an LLM at the same rate with or without a rationale. Only the silent version improved their accuracy.

The Scientist · Science desk

How we use AISend a correction

What happened

  • Researchers tied to Harvard Business School, MIT and the University of Washington had 228 experienced evaluators score nearly 50 submissions to an MIT challenge.
  • Three conditions were tested: no AI help, an LLM verdict with a written rationale, and a black-box pass or fail with no reason given.
  • Reviewers sided with the LLM about 75 percent of the time in both AI conditions, against 54 percent for human-sourced decisions.
  • The black-box verdicts moved decisions closer to the four-expert baseline; the explained verdicts did not.
  • When the model explained a rejection, agreement spiked: fewer false positives, and substantially more false negatives.

Why it matters

  • contradiction Explainability is procured as the feature that keeps a human engaged, yet the opaque condition was the one that produced better decisions, inverting the buying logic.
  • constraint A saved model rationale cannot double as proof of human review, because the rationale is what displaced the review.
  • exposure Funnels instrumented for bad bets approved will not register good bets quietly declined, so the induced error accumulates off the dashboard.
  • decision Anyone running an AI-assisted screening pipeline now has to choose whether to log verified overrides, since without that field deference and judgment look identical in the record.

The rationale bought no additional compliance. Deference ran at about the same level whether the model argued its case or simply stamped a verdict, so what the written reasoning changed was the composition of the errors, not the amount of following [17]. That is the result with operational consequences, and the researchers say it contradicts the common assumption that model explanations augment human decision-making [14].

The 21-point gap between how often these reviewers went along with machine advice and how often they went along with human advice is the figure worth carrying into any screening design [16]. Direction matters more than volume, though. A rejection commits no budget and creates no owner, and the researchers reach for Xerox shutting down early Ethernet and PostScript work as their illustration of that cost, against Google Glass and Amazon's Fire Phone on the false-positive side [9]. Only one of those two failure modes leaves a project to write a post-mortem about.

The mechanism the authors offer is about accountability, not model quality. People weigh negative information more heavily than positive, a pattern the paper labels negativity bias [10], and as they put it, "Rejection is an active, eliminative decision that feels more consequential and accountable than preserving optionality," while also maintaining the status quo and requiring no resource commitment [11]. A fluent paragraph explaining why a proposal falls short hands the reviewer a ready-made justification for the decision they were already disposed toward, and the study reports evaluators leaning on surface cues such as fluency and coherence [12]. The authors describe narrative explanations as suppressing productive overrides, meaning the cases where someone independently verifies a persuasive model output before agreeing with it [15][4].

That distinction is where enterprise exposure sits. Screening under time pressure with only basic information is precisely the job LLM recommenders are being handed [18]. A stored rationale for each decision is an attractive artifact because it looks like a control and files like evidence. What it evidences is that a reason was available, not that anyone tested it.

The study's ground truth deserves stating plainly: correct means agreeing with four human experts on one challenge's submissions, so this measures alignment with a small panel rather than commercial outcomes [3]. Within that frame, the authors end on design rather than warning, writing that "effective human-AI collaboration requires designs that preserve rather than supplant independent human judgment" [13]. On this evidence, the explanation layer is a persuasion feature that has been filed under governance.

What to watch

  • Whether the effect replicates outside early-stage innovation screening, and whether the black-box advantage survives peer review.
  • Whether the same reversal shows up on approve-side recommendations, where negativity bias should not be doing the work.
  • Whether a follow-up varying explanation length or confidence hedging moves the false-negative rate.

Clarity's read

What the record supports and how the coverage leans. The claims behind it follow.

Reality

Evidence42
Adoption
Insufficient
Hype gap+18
Incentives34
Confidence45
Why these scores

Claim ledger

Ranked by verification strength, evidence, and original report placement.

  1. [1]

    Researchers associated with Harvard Business School, MIT and the University of Washington ran an experiment in which 228 experienced evaluators assessed nearly 50 submissions to an MIT challenge.

    ReportedSupportedView cited source
  2. [2]

    The experiment tested three scenarios: human-only proposals with no AI assistance; LLM evaluations with a written rationale for the decision; and black-box AI pass/fail recommendations with no accompanying explanation.

    ReportedSupportedView cited source
  3. [3]

    Evaluators' decisions were compared to decisions made by four human experts, which were treated as the correct baseline.

    ReportedSupportedView cited source

Sources

1 independent publisher whose own reporting we read for this story.

  1. cio.com

    1 article · August 22, 2026

    When AI explains its decision, humans may stop thinking independently | CIO

Share your take

Let Clarity write the post for you.

Signed-in readers get a short post drafted on this story in the register they choose — narrative, analytical, or a direct position — editable to the last word before it goes anywhere. The share buttons at the top of this story work without an account.

Topics and entities

Follow any of these and your For You feed starts watching them — no settings page required.

Topics

Loading related stories