The Correlation That Would Not Go Away

In 1904 Charles Spearman noticed something in schoolchildren's marks that has refused to go away. Performance on unrelated mental tasks correlates positively: people who do well on one tend to do well on others, even when the tasks share nothing obvious. Vocabulary and spatial rotation. Arithmetic and reaction time. Almost any pair you choose.

That pattern is called the positive manifold, and it is among the most replicated findings in the whole of psychology. It has survived a century of attempts to make it disappear, across cultures, instruments and test designs. It is not in dispute by anybody who has looked at the data.

Extract the shared variance from those correlations by factor analysis and you get a general factor, written g. And here is where nearly every argument about this subject goes wrong, in both directions: g is a statistical summary of a robust pattern, not a measured substance. Factor analysis takes a correlation matrix and produces a component accounting for the shared variance; it does not thereby discover an entity. Treating g as a quantity of something in the head, which can be more or less abundant, is a category error, and it is committed constantly, including by people who would explain the distinction correctly if asked directly.

What makes this more than a pedantic point is that the pattern is compatible with several quite different underlying stories. A common processing resource is one. Shared environmental inputs that lift everything at once is another. And the mutualism model is a third: cognitive abilities reinforcing each other during development, so that better vocabulary supports better reading, which supports better knowledge, which supports better reasoning (correlations emerging over time from independent starting abilities rather than reflecting a common cause). It fits the developmental data roughly as well, which is uncomfortable for anybody who has been treating the factor as a discovered thing.

Then there is prediction, where both popular positions are wrong in opposite directions. Cognitive tests predict school and job performance at levels that are genuinely above chance, replicate reliably, and are not artefacts of education or social class. That is unwelcome to one camp. They also leave the large majority of individual variation unexplained. That is unwelcome to the other.

Holding both facts together is the whole skill here, and it has a practical consequence. A measure can be useful for describing a population and nearly useless for ranking particular people. An employer selecting on a cognitive test will, on average, get a group that performs better; any two individuals a few points apart are essentially indistinguishable on the evidence. Using a group-level regularity to make fine-grained individual decisions is the characteristic error of this entire literature, and it appears in almost every applied use of it.