We can start from some old ideas, still much debated.
The Turing Test. The first is from the great mathematician, logician, and pioneer of computer science, Alan Turing. Instead of that question, he suggested the ‘imitation game’: discover by questioning within a time limit whether the tested system can make the tester think it is human. He said, ‘in about fifty years’ (from 1950) ‘it will be possible to program computers ... to play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning’. Some programs surely pass that test.
The Chinese Room argument. But is the test sound? Notice it only takes account of behaviour, ignoring how the behaviour is produced. John Searle’s Chinese Room argument aimed to prove that what goes on inside the system matters. Suppose a certain program enables a computer to understand Chinese, and that there are rule-books in English enabling Searle to do just what the computer does. If he has these rules in a room, then when written Chinese is put through his letter-box, they should enable him (in time!) to put out acceptable Chinese responses – and to understand Chinese. But he insists he doesn’t understand it. The characters remain meaningless to him. He concludes that because the computer does what he does, it doesn’t understand Chinese either. The two main responses to his argument are the Systems Reply and the Robot Reply; he has replied (inadequately I think) to both.
Brute force v. intelligence. Contrast two types of programs for playing board games. One, ‘brute force’, either anticipates every possible position and puts out a fixed response, or else follows an algorithm (e.g. the one by which you never lose at noughts and crosses). Such a program would surely not produce intelligence. The other type may be said to enable the system to weigh up the pros and cons of possible moves – seeming to give it a degree of intelligence.
Block’s machines. Ned Block describes a theoretically (but not practically) possible brute force system producing intelligent responses to questions for e.g. Turing’s five minutes. It has only ‘the intelligence of a toaster’.
What matters? Suggestion: intelligence requires the system to work out its own behaviour using its own assessment of its situation. (Do LLMs do that?)
Block, N. (1981), ‘Psychologism and Behaviourism’;
Searle, J R (1980), ‘Minds, Brains, and Programs’;
Turing, A M (1950), ‘Computing Machinery and Intelligence’.