A Million Examples Prove Nothing

Every other discipline runs on evidence. A drug is tested, the results come in, and at some point the accumulated weight of them makes doubt unreasonable. Nobody has seen the inside of the sun and the physics is not in question. This is how knowledge works almost everywhere, and it is not how mathematics works at all.

Take a claim about all whole numbers. Check it for one, two, three. Check it for the first million. Get a computer and check it to a trillion. You have established the claim for the numbers you checked. About the rest you know exactly what you knew before you started, which is nothing, because the untested cases are not a sample of a population. They are infinitely many specific claims, each of which holds or fails on its own terms.

The standard objection is that this is pedantry. A pattern holding for a trillion cases is surely not going to break at a trillion and one.

Sometimes it breaks.

Pólya conjectured in 1919 that among the numbers up to any point, at least half have an odd number of prime factors. It holds for every number anyone tested by hand, and it holds for every number anyone tested by machine for the next forty years. The first counterexample is 906,150,257. No amount of trying harder was going to find that; it took an argument about why the pattern had to fail somewhere, and then a search in the region the argument pointed at.

The Mertens conjecture is worse. It was disproved in 1985 without anyone producing a counterexample at all. The proof shows one exists; the smallest is somewhere above ten to the fourteenth and nobody has seen it. Here is a statement known to be false, for which no false instance has ever been exhibited.

Fermat's own record is instructive, since he was as good at this as anyone has ever been. He observed that 2^(2^n) + 1 is prime for n up to 4 and conjectured it always was. Euler factorised the next one. The pattern had held for every case Fermat could compute, and computation was the whole of his evidence.

So the asymmetry. One counterexample kills a universal claim permanently, in an afternoon, and no quantity of confirmations establishes one. Disproof is cheap and proof is expensive, and a conjecture can therefore stand for three hundred years and then die on a Tuesday.

What is proof, then, that examples are not?

Here is a claim: add up the first n odd numbers and you always get a perfect square. 1 = 1. 1 + 3 = 4. 1 + 3 + 5 = 9. Keep going and it keeps working, and you can spend an hour on it and learn nothing about why.

Now draw it. Put down one square. Wrap three squares around it in an L shape and you have a 2×2. Wrap five more around that and you have a 3×3. Every odd number is exactly the shell that takes one square to the next, because the shell is a row plus a column plus the corner: n + n + 1, which is odd, and which is every odd number as n runs on.

That is a proof, and notice what it did. It did not raise your confidence. It gave you a reason, and the reason applies to every n at once, including the ones nobody will ever write down. The picture covers infinitely many cases in one page because it identifies the mechanism rather than the instances.

This is the currency the subject trades in and it is why the vocabulary is so careful. A conjecture is a statement believed and unproved. Goldbach's has been checked past four times ten to the eighteenth and is a conjecture, with precisely the status it had in 1742. Mathematicians write entire papers assuming the Riemann hypothesis, and they say so in the first line, because a result proved on an unproved premise is a conditional result and pretending otherwise would be a lie about what has been established.

None of which makes the computation useless. Checking cases is how you form the conjecture, how you decide whether to spend two years looking for a proof or an afternoon looking for a counterexample, and how you find the shape of the thing you are trying to prove. Experimental mathematics is a real and productive activity.

It simply never finishes. The computation tells you where to look. It cannot tell you that you have looked everywhere, and in mathematics everywhere is the only place that counts.