A CIFAR working group puts the magnitude back into research uncertainty

Start with the evidence class, because it governs how much weight the conclusions can bear. CIFAR brought together a room of practitioners for a conversation, not a controlled study ^[2]. Sixteen people were in it ^[16], and what came out is agreement among practitioners plus worked examples from their own fields. There's no control wording and no comprehension measure, and the workshop does not include a before-and-after trust score ^[18]. That doesn't invalidate the conclusions so much as mark them as hypothesis-generating, with a central hypothesis that happens to be unusually testable.

Here it is, in the group's own working definition: uncertainty is everything not yet known about the thing you are trying to describe, and how much that missing knowledge could change the answer ^[7]. Read that as two clauses rather than one sentence. The first clause is the hedge, and hedges are cheap to write. The second is a magnitude claim, and it is the clause a reader needs in order to do anything ^[17]. A finding that names what is missing without bounding how much it matters has satisfied the etiquette of caution and none of its function.

The wildfire example is the cleanest demonstration of why that is structural rather than verbal. A wildfire scientist in the room argued that calling fires simply unpredictable risks abdicating responsibility: nobody can say when a given fire starts, how fast it moves or what it takes, but it is near-certain that fires will happen somewhere, and treating the whole thing as unknowable becomes an excuse not to plan ^[9]. The word "unpredictable" is doing the damage by collapsing a specific, bounded ignorance into a general one. Meanwhile the planning obligation does not move; local authorities still have to build mitigation on the assumption that a fire is coming ^[10].

The BICEP2 case cuts the other way, and it is the more disciplining of the two. In 2014 a team announced it had glimpsed the universe's first instant, and the signal turned out to be consistent with dust in our own galaxy ^[14]. Physics demands extraordinary confidence before it will say discovery, which the announcement had. What confidence about a number does not cover is whether you are measuring the right thing at all ^[14]. That's a category error, not a statistical one, and no error bar can capture it.

Which is why the statistician's point in that room lands harder than it sounds: the line between significant and nonsignificant, conventionally drawn with a p-value at 0.05, is as much a social convention as a mathematical one ^[13]. The thing a threshold crossing does not tell you is how big the effect is, or whether the measurement was pointed at the quantity you care about.

So the reader's tests fall out of the group's own categories: whether the account states a magnitude of doubt or only its existence, whether it distinguishes "we cannot say when and where" from "we cannot say anything," and whether it says whether the result was modeled in the lab or observed in the world, which the group flags as its own species of uncertainty, along with not knowing how people will respond to the information ^[11]. Genetic counselors do this translation as their job, moving population-level risk into something an individual can decide with ^[12], and an investigative journalist made the same distinction from the other end: beyond a reasonable doubt is not absolute certainty, and wrongful convictions happen when a court forgets the difference ^[15].

My view, with its condition attached: I would trust a result that states how wrong it could be over one that clears 0.05 quietly. The condition is that this comes from experienced communicators reasoning together ^[1], not from any measurement of what readers actually understood.