The Schedule That Built the Slot Machine

Start with the terminology, because it is misused so consistently that the misuse gets in the way of the ideas.

Reinforcement increases a behaviour. Punishment decreases it. Positive means something is added; negative means something is taken away. That is all the words mean: "negative" is subtraction, not unpleasantness.

So negative reinforcement is removing something aversive in order to increase a behaviour. Taking a painkiller removes pain, which reinforces taking painkillers. Fastening a seatbelt stops the alarm, which reinforces fastening seatbelts. Leaving a party where you feel anxious removes the anxiety, which reinforces leaving.

That last example is why this matters clinically. Escape and avoidance are negative reinforcement, and they are the engine underneath most anxiety maintenance: the relief of getting out is immediate and powerful, it reinforces getting out, and the person never discovers that staying would also have been survivable. Every avoidance strengthens the pattern. Calling this "negative reinforcement" when people think the phrase means "punishment" is one of the reasons the mechanism is so widely misunderstood.

The second definitional point is stranger and more useful. A reinforcer is defined by what it does to the behaviour, not by what the person delivering it intended.

If a consequence does not increase the behaviour, it was not a reinforcer, whatever it was meant to be. A teacher's sharp reprimand delivers attention, and for a child who has little of it, attention can reinforce the disruption it was meant to suppress. A bonus scheme that changes nothing was not a reinforcer. The definition looks circular and is deliberately constructed to force the question to be settled by measurement rather than by assumption about what people ought to find rewarding.

Now the finding with the largest commercial footprint of anything in psychology.

Reinforcement can be delivered on a schedule. Every time; every nth time; after a fixed interval; or (the important one) on a variable ratio, where the reward comes after an unpredictable number of responses averaging some value.

Variable ratio produces high, steady rates of responding and is extraordinarily resistant to extinction. The reason is straightforward once stated: under a predictable schedule, a missed payout is immediate evidence that something has changed. Under variable ratio, long runs without reward are a normal feature of the schedule, so there is no point at which the evidence tells you to stop. The dry spell you are in is indistinguishable from the dry spells that ended in a payout.

Every slot machine runs on this. So do loot boxes, so does pulling to refresh a feed, and so does checking whether anyone has replied. It is the single most exploited finding in behavioural psychology, and it was characterised in pigeons in the 1950s by people with no commercial interest whatsoever.

The constructive side of the same toolkit is shaping, and it deserves more attention than the schedules.

Complicated behaviours never occur spontaneously for you to reward. You could wait forever for a pigeon to bowl or a rat to pull a lever in a particular sequence. So you reinforce anything in roughly the right direction, then raise the criterion slightly, then again. Each step is small enough that the learner succeeds often enough to keep going.

That is also the entire design principle behind graded exposure, well-made tutorials, physiotherapy and any skill acquisition programme worth the name. When people fail to learn something, the commonest fixable cause is that the step size was too large, so success was too rare to sustain the effort.

Which leaves punishment, and the evidence is less encouraging than intuition expects.

Punishment does suppress behaviour, in the presence of the punisher, and it teaches nothing about what to do instead. Its side effects are substantial and well documented: avoidance of the punisher rather than of the behaviour, aggression, and a learned association between the whole situation and fear. It can work when it is immediate, consistent, and paired with a reinforced alternative: a combination that is rare in practice, because real punishment tends to be delayed, intermittent and delivered by someone who is angry.

Reinforcing an incompatible behaviour is almost always the better instrument, and it has the advantage of specifying what you actually want.

One last thing that will save somebody a bad week. When a previously reinforced behaviour stops being reinforced, the first response is escalation: do it harder, longer, louder. This is the extinction burst, and it is entirely normal, because escalating has often worked before.

It is also the point at which most people give in. And giving in after an escalation does not merely restore the old pattern. It delivers reinforcement on a variable ratio schedule, at a higher intensity, which makes the behaviour far more persistent than it was before anybody tried to change it. If you are going to run an extinction procedure, the burst is the part you have to plan for.