The Test Is 99% Accurate

A disease affects one person in ten thousand. There is a test for it, and the test is right ninety-nine per cent of the time. You take it. It comes back positive.

What is the chance you have the disease?

Most people, including most doctors when the question is put this way, say ninety-nine per cent, or something close. The answer is about one per cent.

The sum is easier than the intuition. Imagine ten thousand people like you. One of them has the disease, and the test almost certainly catches them: one true positive. The other 9,999 are healthy, and the test is wrong one time in a hundred, so about a hundred of them test positive anyway. That is a hundred false positives.

A hundred and one positive results. One of them is real. Your chance of being that one is about one in a hundred.

Nothing here is a trick. The ninety-nine per cent figure is true and is a statement about the test: given that you have the disease, the test says so. The question you care about is the other direction: given that the test says so, do you have the disease? Those are different quantities, and they come apart whenever the condition is rare, because rarity means there is an enormous healthy population available to be wrong about.

Improving the test does not rescue you. Make it right 999 times in a thousand and you still get about ten false positives for every true one at that prevalence. The dominating term is not the error rate; it is the size of the healthy group. This is why screening programmes are argued over rather than simply expanded, and why recommendations tighten as well as loosen. Every false positive carries follow-up investigation, weeks of fear, sometimes a biopsy, and sometimes treatment for something that was never going to cause harm.

The same error has a name in court, where it does real damage. A forensic match occurring in one person in a million is not a one-in-a-million chance that the defendant is innocent. In a city of ten million, ten people match. Which of them is on trial, and why, are questions the match cannot answer. This is the prosecutor's fallacy, and it has helped convict people who were later cleared.

The Sally Clark case is the one usually cited. Two of her children died in infancy; an expert witness gave the chance of two cot deaths in one family as one in seventy-three million. The figure was calculated by squaring a single-death rate, which assumes the deaths are independent when everything about shared genetics and shared environment says they are not. And it was heard by the jury as the probability that she was innocent, which it was not. The Royal Statistical Society wrote publicly about the error. She was released after three years and died four years later.

Here is the encouraging part. The difficulty is in the presentation, not the mathematics.

Gigerenzer and colleagues gave doctors the standard percentage version and most got it wrong. Then they gave them the same problem as counts: out of ten thousand people, one has the disease, and about a hundred healthy people test positive. Most got it right. The arithmetic is identical. What changed is that the count version carries the base rate visibly through the calculation, where percentages normalise each group separately and quietly drop the thing that matters.

That result has changed how risk is taught in medical schools and how some screening leaflets are written. It is one of the clearest cases of a finding about human reasoning producing a practical fix, and the fix is not to train people harder. It is to present the numbers in the form that keeps the population in view.

The general lesson is worth stating plainly, because it applies far beyond medicine. Whenever you are told how accurate something is (a test, a scanner, a fraud detector, a facial recognition system) the accuracy figure alone tells you almost nothing about what a hit means. You need to know how rare the thing being detected is. When it is rare, most hits are false, however good the instrument.