Getting a Number · Chapter 3 of 3
Instruments Make Phenomena
On why a reading is only as good as the theory behind the device.
3 min read
An instrument does not passively reveal what is out there. It embodies a chain of reasoning: from the thing being measured, through a physical interaction, to something a human can read. A telescope image is light interpreted through optics, a detector response and processing software, each step carrying physical assumptions. So the reading is only as good as that chain.
This is the point philosophers call theory-ladenness, and it is easy to hear as a sceptical complaint. It is not one. The assumptions in a well-understood instrument are usually excellent and heavily tested. The point is practical: when a reading surprises you, the fault is at least as likely to be in the chain as in the world, and knowing that is what distinguishes a discovery from an artefact.
Everything that experimental practice does about this follows from taking it seriously. Calibration tests the chain against known standards. Independent confirmation tests it against instruments with different assumptions. Modelling what the apparatus would do if it were wrong (what signal a fault, a contaminant or an overlooked source would produce) is how you find out whether your result could have arisen some other way.
The first direct detection of gravitational waves is the clearest illustration available. The measurement involved a length change smaller than the width of a proton across a four-kilometre arm. Nobody would believe that on the word of one instrument, and the collaboration did not ask anyone to. Two detectors were built three thousand kilometres apart, so that neither could confirm itself. A truck, an earthquake, a maintenance error appears in one. A passing wave appears in both, with a light-travel-time delay that also constrains the direction it came from. Independent confirmation was not a refinement here; it was the precondition for anyone believing the result at all.
There is a second, subtler defence, and it addresses the analyst rather than the apparatus. In a blind analysis the procedure is fixed before the result can be seen: by adding an unknown offset to the data, by working on a scrambled subset, or by registering the full analysis publicly before unblinding. The problem it solves is not dishonesty. It is that everyone stops checking when the number matches expectation and keeps looking for errors when it does not, and no amount of good intention removes that asymmetry. Only a procedural commitment does.
The instructive failure is BICEP2. In 2014 a collaboration announced the detection of primordial gravitational waves from the early universe, visible as a polarisation pattern in the cosmic microwave background. The announcement was withdrawn when better data showed the pattern could be produced by dust in our own galaxy.
Nothing was wrong with the instrument. The measurement was competent, the analysis careful, the signal real. What was incomplete was the model of what else could produce it. That is a much harder failure to guard against than a broken detector, because every internal check passes. The only defence is asking, persistently and before announcing anything, what else in the world could have made this reading, and then going and measuring that.
The detector moved by a fraction of a proton's width. Believing that required believing the whole instrument.
Where you'll meet this
Two detectors three thousand kilometres apart were built partly so that neither could confirm itself. A local disturbance shows up in one; a passing wave shows up in both with the right delay.