Building a Basket

A price index prices a basket of goods in two periods and reports the ratio. The concept takes one sentence. Every difficulty is in the assumption that the basket stays fixed, and the difficulties are large enough to have moved hundreds of billions of pounds and dollars between governments and citizens without anybody voting on them.

Start with substitution. If beef doubles in price and households shift to chicken, they are worse off, but by less than the beef price implies, because an alternative existed. An index that reprices last year's basket every year assumes nobody switched, and so overstates the cost of remaining equally well off. Chained and superlative index formulas update the weights to capture the shift. The choice between formulas produces a persistent gap of several tenths of a percentage point a year, which compounds.

Then new goods. A product typically launches expensive and falls steeply as production scales, but it enters the basket only once spending on it is large enough to register. The early decline, where most of the consumer gain sits, is therefore never captured at all. Agencies have shortened the lag considerably and cannot eliminate it without pricing goods almost nobody buys.

Then quality. The same nominal price can buy a materially different object, and separating the quality component from the pure price component requires a hedonic model: a regression of observed prices on observed characteristics, whose coefficients are then used to value whatever changed between versions. It is indispensable for rapidly improving goods and it is genuinely a model, which means the published inflation rate depends on specification choices made inside a statistical agency and visible to almost nobody outside it.

In 1996 the Boskin commission put numbers on all of this for the United States and concluded that the consumer price index had been overstating inflation by roughly 1.1 percentage points a year. Compounded over a working life the implication is substantial: real wage growth across the preceding decades had been correspondingly understated, and the stagnation everybody was arguing about was partly a measurement artefact. The methodology was contested (an overstatement conveniently reduces indexed spending, which critics noted) and several recommended fixes were adopted anyway.

Why does a technical dispute of this kind matter so much? Because these indices are load-bearing. Pensions, benefits, tax thresholds and index-linked government bonds are all tied to them. A decision about formula or weighting flows straight into what the state pays and receives, at a scale that dwarfs most announced budget measures, with no equivalent scrutiny attached. Overstate inflation by a point a year and you understate real wage growth by a point a year for thirty years, and then everyone argues about the resulting narrative rather than the arithmetic underneath it.

The last thing worth saying is that there is no single true rate waiting to be found by better methods. A pensioner buying medicine and heating and a young renter in a city face genuinely different price changes, and the published average describes a composite household that does not exist. Better sampling cannot fix this, because it is not an error. It is what averaging across different lives necessarily means.