Topic

Flaky and Nondeterministic Tests

Tests whose outcome varies between runs for reasons unrelated to the code under test, including hash ordering, process-level randomisation and timing.

Current clusters

build1 publisher

Ten packages each ship a test that would declare the package unnecessary

An exit status of 0 covers both a suite that passed and a suite that never ran, so ten new verification tools each commit a control test that fails if the tool's own reason for existing has gone away.

Publishers:dev.to

Reality

Evidence55
Adoption12
Hype gap+12
Incentives65
Confidence50