Measured Precisely, Not Understood

In the 1970s Vera Rubin measured how fast stars orbit at the edges of spiral galaxies, expecting them to be slower than stars near the centre. This is what the solar system does: Neptune takes 165 years to orbit while Mercury takes 88 days, because gravity weakens with distance from the mass.

The outer stars were moving as fast as the inner ones. Galaxy after galaxy, the rotation curves went flat.

Either galaxies contain far more mass than we can see, arranged in an extended halo, or gravity does not behave as expected at those scales. That is the whole problem, and it has not been resolved in fifty years.

Rubin's measurements were not the first hint (Fritz Zwicky had found something similar in galaxy clusters in 1933 and been largely ignored) and they were precise enough and numerous enough that the discrepancy could not be dismissed.

Since then, independent lines of evidence have converged on the same answer. Gravitational lensing measures the mass of a cluster by how much it bends light from behind it, and finds more mass than the visible matter. The cosmic microwave background's fluctuation spectrum depends on the relative amounts of ordinary and non-interacting matter, and fits a specific ratio. The formation of large-scale structure requires extra gravitating material to have started clumping before ordinary matter could.

Four different methods, measuring different things at different scales, agreeing on roughly the same amount. That convergence is the main reason most physicists prefer extra mass to modified gravity.

The alternative deserves a fair hearing, because it is better than its reputation. Modified Newtonian dynamics, proposed by Mordehai Milgrom in 1983, adjusts gravity's behaviour at very low accelerations, and it fits individual galaxy rotation curves remarkably well: in some respects better than dark matter, which requires fitting a halo to each galaxy separately. It predicted a relationship between a galaxy's visible mass and its rotation speed that was later confirmed.

Where it struggles is at cluster scales and with the microwave background, and there is one observation that is particularly hard for it.

In the Bullet Cluster, two galaxy clusters have collided. The hot gas, which is most of the ordinary mass, slowed down through collision and sits in the middle. The galaxies themselves, being mostly empty space, passed through and are out ahead. And gravitational lensing shows the bulk of the mass is out ahead too, with the galaxies, not in the middle with the gas.

That is what collisionless dark matter would do. For a theory with no extra mass, in which gravity simply follows the visible matter, it is a difficult observation, because the mass and the visible matter are in different places.

The other 68% of the universe's energy content is stranger still and was a complete surprise.

In 1998 two teams measured distant type Ia supernovae in order to work out how much gravity was slowing the expansion. Everyone expected deceleration; the question was how much. The supernovae came back dimmer than they should have been, meaning further away than expected, meaning the expansion is speeding up.

Both teams checked for every systematic they could think of, found it independently, and shared a Nobel in 2011. Dark energy is the name attached to whatever is causing it.

Naming is not explaining. The leading candidate is a cosmological constant (an intrinsic energy density of empty space) which fits the data well and has a serious theoretical problem: quantum field theory's estimate of the vacuum energy exceeds the observed value by many dozens of orders of magnitude, which is sometimes described as the worst prediction in the history of physics.

Meanwhile, decades of increasingly sensitive underground experiments have failed to detect a dark matter particle, and have now excluded much of the parameter space where the favoured candidates were expected. This is a genuine problem rather than a technicality.

So around 95% of the universe's content is measured precisely and not understood. It is tempting to conclude that cosmology is therefore guesswork.

The conclusion does not follow, and the reason is worth understanding. The standard cosmological model has six free parameters and predicts the detailed fluctuation spectrum of the microwave background, the abundances of the light elements, the statistics of how galaxies are distributed across the sky, and the expansion history: all matching observation closely. A theory can describe how something behaves with great precision while not knowing what it is.

Newton's gravity was exactly that for two centuries. It predicted the planets to extraordinary accuracy, and Newton himself declined to say what gravity was, on the grounds that he had no hypothesis worth offering.