Skip to content

Science1 publisher3 min readPublished

A New Scientist column tests p(doom) against three risks that can be counted

Estimates of the chance AI kills everyone run from zero to more than 95 per cent, so the column sets that range beside risks whose numbers come from counted lottery tickets, WHO death tolls, climate model ensembles and asteroid surveys.

The Scientist · Science desk

Illustration accompanying A New Scientist column tests p(doom) against three risks that can be counted

What happened

  • A New Scientist column points out that leading commentators' p(doom) estimates run from 0 to more than 95 per cent, a range that covers the entire probability scale.
  • The column's position is that no one can produce a scientifically meaningful probability of an AI apocalypse, because the question is not a scientific one.
  • For asteroids, it cites NASA's finding that more than 90 per cent of bodies wider than 1 kilometre have been catalogued, and notes their future positions can be predicted.

Compiled by The ScientistSomething wrong?How this is made

Why it matters

  • decision A probability that swings 70 percentage points depending on which policy is assumed can rank policies. An elicited belief about AI extinction does not move with any intervention, so it cannot rank interventions.
  • constraint Before any single risk percentage can be set beside another, it has to name its period and its population: the same WHO toll supports 0.0125 per cent for one year and about 0.87 per cent across 70.
  • precedent Reporting a greater-than-10-per-cent figure without the surrounding spread lets an opinion travel with the authority of a computed statistic, and the next round of coverage will inherit the number without the method.

What separates a computed probability from an elicited one is whether there is something to count. The lottery has a sample space. About 1 in 4.9 UK National Lottery tickets wins some prize, which the column gives as 20.4 per cent, so 79.6 per cent lose [4][5].

Road deaths are messier, and the column is candid about it. The World Health Organization puts annual road traffic deaths slightly above 1 million, and dividing by 8 billion people gives 0.0125 per cent for a single year [6][1]. The column rounds that to roughly 0.01 per cent and then says plainly that it is wrong for any individual [7]. "More than 90% of road traffic deaths occur in low- and middle-income countries", the WHO says, as quoted in the piece [8]. Hold the same annual rate for 70 years and it compounds to about 0.87 per cent, near 1 in 115 [2]. The factor of 70 is the length of the window.

Climate is the harder case, and the column lists why: warming is a continuum, there is only one world to observe, and the 2100 figure depends on choices not yet made [9]. Tested models supply the missing sample by generating many "parallel worlds" whose statistics can be compared across policies [10]. Under current policies, the UN Environment Programme's Emissions Gap Report gives a 100 per cent chance of exceeding 1.5C this century, 92 per cent for 2C and 20 per cent for 3C; on the strongest assumptions of climate action, those fall to 79, 22 and 0 per cent [11][12]. The 2C number moves 70 percentage points between the two cases [3].

The asteroid comparison needs three inputs, which the column names: how many asteroids there are, the chance one hits Earth, and the damage it would do [13]. NASA has found more than 90 per cent of those wider than 1 kilometre, the bigger ones being both easier to spot and more deadly, and their future positions can be predicted [14].

So the 0-to-more-than-95-per-cent spread among leading commentators [3] is evidence about the estimation procedure, not about machines. That width is what a quantity looks like when no shared method exists to derive it: every answer is admissible because none can be checked against anything. None of that means the risk is small. A hazard nobody can quantify has not been quantified as zero, and the same range that holds the zeros holds the 95s [3]. A figure produced this way should not be handled like the UNEP percentages, which arrive with error bars and a stated dependence on policy [11].

One more caution about the headline number. When coverage reports a greater-than-10-per-cent chance that AI could kill all humans [1], the column's answer to what that means is that it means very little, because the question is not a scientific one [2]. The road-traffic example shows what is missing before any such figure can be compared with another: over what period, and to whom.

What to watch

  • Whether the next UNEP Emissions Gap Report moves the 92 per cent chance of exceeding 2C under current policies, and in which direction.
  • Whether anyone publishing a p(doom) percentage states the time window and the population it applies to.
  • Whether NASA's survey completeness for asteroid sizes below 1 kilometre gets reported alongside the above-90-per-cent figure for larger bodies.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories