The Ten Sailors Who Got Nothing

In 1747 James Lind had twelve sailors with scurvy aboard HMS Salisbury. He divided them into six pairs and gave each pair a different remedy: cider, sulphuric acid, vinegar, seawater, a paste of garlic and mustard, or two oranges and a lemon.

The pair on citrus recovered within days. It is usually called the first controlled trial, and what makes it one is not the oranges. It is the other ten men.

Without them, Lind would have had two sailors who got better after eating fruit, and that is not evidence of anything, because the overwhelming majority of what makes a treated group improve has nothing to do with the treatment.

Start with the simplest: people get better. Most conditions are self-limiting, most symptoms fluctuate, and a great many treatments in history owed their reputations to the body's own recovery. Then add regression to the mean. People seek treatment when symptoms are at their worst, which is partly a genuinely bad phase and partly bad luck, and the luck component drifts back toward average whatever anyone does. Any intervention timed to a peak will appear to work.

Then expectation. The placebo effect is real for subjective outcomes and much weaker than folklore claims for objective ones. It will not shrink a tumour, and it will reliably move a self-reported pain score. And then the simple fact of being in a study: people who know they are being observed take their medication, attend appointments and behave differently in ways that improve outcomes regardless of what they were given.

Every one of these gets credited to the intervention if there is nobody to compare against. Which is why uncontrolled before-and-after studies are so consistently encouraging and so consistently wrong.

The control group handles all of it in one move. But it only works if the two groups are otherwise comparable, and that is what randomisation is for.

The temptation is to match: assign by age, severity, sex, and so on, so the groups look alike. The trouble is that this only handles the confounders you thought of, and the ones that wreck a study are the ones nobody named. Randomisation distributes every characteristic (measured, unmeasured, and unimagined) equally in expectation. That is the whole reason a randomised trial licenses a causal claim without a theory of what else might be going on, and no amount of statistical adjustment in an observational study reproduces it.

Blinding closes the next gap. If participants know which arm they are in, expectation does its work, so they are blinded. If the person assessing the outcome knows, they will score an ambiguous result in the direction they expect, without any intention to, which is the half most often neglected, and the half that matters most for subjective endpoints. Hence double-blind: neither the participant nor the assessor knows. Where blinding the participant is impossible, as in surgery or most behavioural interventions, the assessor can still be kept in the dark, and a study that does not is much weaker than one that does.

Two procedural points remain, and both are about what happens after the data exists.

The first is intention to treat. Some participants drop out, stop taking the drug, or take it badly, and there is a strong urge to analyse only those who completed the protocol, surely that is the truer test of the drug. It is not, because people stop for reasons connected to the treatment: side effects, feeling better, a regimen they cannot manage. Removing them breaks the randomisation and leaves a self-selected group that is no longer comparable to the controls. Analysing everyone in the arm they were assigned to is conservative, and it answers the question a patient actually has, which is what happens if this is prescribed rather than what happens if it is taken perfectly.

The second is registration. A trial specifies its primary outcome in advance, in a public registry, with a timestamp. This sounds bureaucratic and closes a large hole: before it was required, a trial could miss its stated endpoint, find that some other measure had reached significance, and publish that instead. Studies comparing registered protocols with published papers still find discrepancies in a substantial fraction of trials. Registration adds no data. It makes the analysis auditable, which turns out to be the more valuable thing.

None of this means observational evidence is worthless. It is indispensable where randomisation would be unethical or impossible (nobody is going to randomise people to smoke) and the case against tobacco was built from observational data, carefully, over decades, using consistency across designs, dose-response, and biological plausibility. But the reason it took decades is exactly the problem: coffee drinkers differ from non-drinkers in income, smoking, sleep and occupation; adjustment only touches what was measured; and people in early ill health often change their habits, so the arrow can point backwards. A correlation survives adjustment for many reasons besides being causal.