Topic

Sampling and measurement uncertainty

Treating a metric as an estimate from a sample, with confidence intervals and run counts that determine how precise it can be.

Current clusters

security1 distinct publisher

A METR author puts an order-of-magnitude error bar on the time horizon curve

The 50% time horizon measures replaceable serial human labour, not autonomous runtime, and the published interval on the frontier measurement is about as wide as a year of the curve it sits on.

Publishers:metr.org

Reality

Evidence74
Adoption30
Hype gap−22
Incentives40

build1 distinct publisher

Thirty runs of one prompt put a 34-point error bar around a brand's mention rate

A Search Engine Land analysis published in February 2026 found that repeating the same ChatGPT prompt returns different brands, which makes visibility a proportion you sample. The arithmetic of that proportion sets how many runs your report needs.

Publishers:dev.to

Reality

Evidence30
Adoption: Insufficient
Hype gap