build1 publisher
Repeated A/A tests turned up a 30% false-positive rate on one experimentation platform
The miscalibration sat in the stats engine, against a nominal 5%, beyond the reach of correct math downstream. Routing unallocated traffic into a standing A/A test is how a team measures its own rate.
Publishers:dev.to
Reality
- Evidence48
- Adoption
- Insufficient
- Hype gap+12
- Incentives18
- Confidence55