The misses must be counted

An exciting result needs the quiet results beside it.

FIGURE 11 / Made-up test: 10 of 1,000 stones hold gold. It finds 9 of those, but wrongly flags 99 others. Only 9 of its 108 positive results hold gold: about 8.3%.

Of 1,000 stones, 10 contain gold. A test detects 90% of gold stones and falsely flags 10% of the others.

Nine true positives and 99 false positives are expected. Only 9 of 108 positive results contain gold: about 8.3%.

The arithmetic is conditional on the stated rates. Real test performance must be measured in the relevant population.

For humans, an impressive test result can still be misleading when the target is rare. Misses and false alarms must also be counted.

The success is noted. The other 99 stones would like a word.

108 positive tests: 9 contain gold; 99 do not