What Review Actually Catches

Peer review is younger than almost anyone assumes. Editors historically made publication decisions themselves, consulting a colleague when they felt out of their depth. External review became a universal requirement only after the Second World War, driven by growing specialisation and the administrative needs of funding bodies. Watson and Crick's paper on the structure of DNA was not externally reviewed, and neither were a great many of the results that define twentieth-century science.

What the process consists of is worth stating plainly, because the public image and the reality have drifted apart. Two or three specialists read a manuscript. They generally do not have the underlying data. They never repeat the experiments. They are unpaid volunteers giving a few hours to a paper in a week that already had other demands. They see what the authors chose to show them.

Given that, they do a specific job reasonably well. They catch unclear argument, missing controls, conclusions the results do not support, relevant prior work that has been ignored, and statistical treatments that do not match the design. Papers usually improve between submission and publication. It is a competence check by informed readers, and at that it works.

What it cannot do is detect fabrication. Nothing in the process would reveal invented numbers, and expecting otherwise asks a reader with no data and no laboratory to perform an audit. Much of the public disillusionment with peer review follows from this misunderstanding rather than from any decline in how it is practised: it is being judged against a job it was never structured to do.

There is a second limitation with larger consequences for the literature as a whole, and it sits upstream of review entirely. A journal's most consequential judgement is usually not accept or reject but what counts as interesting. Null results and replications are hard to place in prominent venues, so the published record is a biased sample of the research actually conducted. An effect absent from nine unpublished studies and present in the tenth enters the literature looking like a finding. Registered reports, where a paper is accepted on the strength of its design before any results exist, are a direct structural response to this and are spreading slowly.

Preprints change the timing rather than the substance. Posting a manuscript publicly before review moves scrutiny after publication and widens it enormously: more eyes, faster, from anyone who cares. Physics has worked this way for over thirty years with no evident collapse in quality. Biology and medicine adopted it during the pandemic and have largely kept it.

The trade is real. A public record that includes work which does not survive scrutiny is a record from which journalists and campaigners will quote. Among specialists this works, because preprint status is visible and the surrounding commentary is informative. Once a claim escapes into general circulation, the caveat falls off first.

None of which argues for abandoning review. It argues for describing it accurately: a useful filter applied by a few readers with limited information, embedded in a system where replication and post-publication scrutiny do the heavier work.