Skip to content

Leadership1 publisher2 min readPublished

Graduate-degree viewers scored below a coin flip at telling a human video edit from an AI one

Graduate-degree viewers in a 315-person study identified the human-edited video 34.8% of the time, below a coin flip. The result rests on one pair of videos where the human editor added the polish, so what carries over is the advice to test content on real users.

The Board Room · Leadership desk

Illustration accompanying Graduate-degree viewers scored below a coin flip at telling a human video edit from an AI one

What happened

  • A study summarized in an Entrepreneur contributor column showed 315 people one piece of raw footage edited twice, by a professional human editor and entirely by AI, and asked which was which.
  • Graduate-degree holders guessed correctly 34.8% of the time, against 51% for bachelor's holders and 45.5% for respondents with a high school education.
  • Accuracy fell as household income rose, and the highest earners in the sample were the least likely to pick out the human-edited video.
  • Most respondents took plain editing for human work and polish for AI, but here the human editor added the branded motion graphics and captions while the AI kept things minimal.

Compiled by The Board RoomSomething wrong?How this is made

Why it matters

  • exposure A company paying human editors for branded graphics and captions can have that work credited to AI by its highest-earning, most credentialed viewers, because those were the cues this sample took for machine work.
  • constraint Segment-level conclusions from this study are hard to carry to another business, because its result depends on which version happened to carry the polish.
  • decision Moving care away from affluent segments on this evidence swaps one untested rule for another; the defensible spend is a user test on the specific content before any reallocation.

On a two-way choice, random guessing lands near 50%. Graduate-degree holders finished 15.2 points under that line, and bachelor's holders sat within a point of it [1][2]. Scoring well below chance takes a consistent rule aimed at the wrong cue. The column's authors say the higher earners and graduates used the same rule as everyone else, only with more confidence [5].

That explanation comes from the team that ran the study, writing as an Entrepreneur contributor; the publication notes that contributor opinions are their own [10]. The column reports accuracy by group but does not give how many of the 315 respondents fell into each group, or how confidence was measured [1]. The authors say the full methodology and dataset are published separately [6]. Until those subgroup counts are checked, the 16.2-point spread between bachelor's and graduate-degree holders comes from samples of unknown size [3].

The board-deck version of the study is that affluent, credentialed customers are easier to fool. That version leaves out the footage. The polish in this test came from the human editor [4]. If respondents were following a polish-means-AI rule, as the authors argue, a pair in which the AI added the graphics would have rewarded the same people, and the group applying the rule most consistently would have scored best [4]. The result describes one pair of edits of one piece of footage [1].

The authors carry the lesson past video, to any business assuming a customer segment will catch a shortcut or a quality drop [11]. The task, as described, was attribution: say which cut a human made [1]. The decision the authors say this assumption drives is which segment gets the polished version and which gets the rushed one [7]. That choice turns on whether customers notice or mind the cheaper version, and knowing who made a video answers a different question.

What holds up is the authors' account of their own habit. "We had never actually tested whether that assumption holds up," they wrote [8]. For an operator, the trade-off this quarter is a paid test on the specific content against a reset based on a rule of thumb. That rule could be the old one about discerning high earners or the reversed one this column invites. A reset made without a test this quarter leaves next quarter's results measured against a rule nobody checked on the company's own customers. The authors' recommended fix is to test the actual content or product against real people and watch what they do [9].

What to watch

  • Subgroup counts in the published dataset: small graduate-degree or top-income cells would weaken the 16.2-point education gap.
  • A rerun with a pair where the AI edit carries the motion graphics and the human cut stays plain; if graduate-degree viewers then score best, the rule drove the result.
  • Tests outside video, such as written copy or product quality, since the authors claim the pattern extends to any shortcut a segment is assumed to catch.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories