The Chinese Room, and Other Minds

John Searle asks you to imagine him sealed in a room with a very large rulebook. Chinese characters come in through a slot. He looks them up, follows the rules, and passes other Chinese characters back out. The responses are indistinguishable from those of a native speaker. He does not understand a word of Chinese.

The argument is short and its target is precise. All a program does is manipulate symbols according to rules. If manipulating symbols according to rules is compatible with total absence of understanding, then running a program cannot be sufficient for understanding, however good the output. Syntax is not sufficient for semantics. The claim under attack is the strong one (that the right program just is a mind) and the paper says nothing about whether machines could be conscious by some other route.

Indeed Searle's actual position is routinely misreported as the view that no machine could ever be conscious, and he says the opposite. Brains are machines. They cause consciousness by their specific causal powers. A machine with the right causal powers could be conscious. What he denies is that the program is where those powers live, whatever hardware it runs on.

The most popular objection is the systems reply: the man does not understand Chinese, but the whole system (man, rulebook, room, paper) does, in the way that no neuron in your head understands English while you do. Searle's answer is to have the man memorise the entire rulebook and work outdoors, so there is no system left besides him, and he still does not understand. This is the most contested passage in the paper. Critics say it is far too quick: a person who has internalised a vast rule system may well constitute two things at once, and the intuition that he does not is being asked to carry the entire argument on its own.

Behind all of this sits an older and more uncomfortable problem. You have direct access to exactly one mind. Every attribution of experience to anybody else is an inference, and it cannot be checked. The usual inference runs from behaviour together with physical similarity: they wince as I do, they are built as I am, therefore they feel as I do. It is a perfectly good inference, and it has never been tested, because those two considerations have always travelled together.

That is what has changed. A system that produces the behaviour while sharing none of the substrate pulls the two halves apart for the first time, and it turns out that nobody has a principled account of which half was doing the work. That is why a paper from 1980 about a man in a room is suddenly being cited in product reviews, generally by people on both sides who have not read the replies.

It also explains why confident answers about chatbots are so unsatisfying in both directions. A system reports having feelings and describes them in detail. Treating that as evidence assumes behaviour suffices, which is exactly what Searle denies. Dismissing it because the system was trained to produce such text assumes behaviour does not suffice, which is what his critics deny, and people are also trained, by upbringing and by language, to report feelings in the available vocabulary. The case does not discriminate between the positions. Saying so is not a dodge; it is the only honest thing available, and it is a good deal more useful than a verdict nobody can support.