Generative AI is often said to have killed the Turing test. But this paper argues that the machines may not have passed Turing’s test at all—and then puts Turing’s original design into practice. In October 1950, Alan Turing proposed that a machine’s success in an imitation game provides a criterion for thinking.

Turing’s test became one of the philosophical foundations of AI, but three-quarters of a century later, it is widely claimed that generative AI models pass the test, rendering it obsolete. The paper’s aim is to establish definitively Turing’s conceptual design for his game, argue that influential interpretations are mistaken, and present results from carrying out Turing’s experiment as proposed.

In his October 1950 paper in Mind, Turing described an imitation game involving a question and answer method and proposed that success provided a criterion for thinking in machines. That paper contrasted sharply with nearby philosophy-of-mind articles, which discussed topics such as sense data, selves, mental acts, psycho-physical parallelism, and souls, rather than learning machines or artificial intelligence.

Some early replies did not take Turing’s proposal very seriously, and Leonard Pinsky soon proposed an alternative test for intelligence in machines. Three-quarters of a century later, the verdict on Turing’s test is often cavalier; a Nature article claimed that large language models had killed the test because chatbots now ace the imitation game.

The cited evidence was a 2025 study stating that two large language models pass the Turing test, prompting commentators to declare the test obsolete. The vital question is whether these machines passed Turing’s test, or merely some imitation test.

The first aim is to establish definitively Turing’s conceptual design for the test by taking Turing’s descriptions as the basis. The paper argues that influential interpretations are incompatible with Turing’s own words and conflict with the early perception and historical context of his ideas.

The claim that unintelligent large language models have passed Turing’s test, thereby rebutting his criterion for thinking, is false. The second aim is to present results from implementing Turing’s conceptual design for the first time. The study does not support the announced general consensus that AI has passed the test; when a generative pre-trained transformer machine was tested, its performance was extremely poor.

The paper has two parts: the first identifies Turing’s conceptual design, and the second presents a new empirical study implementing that design. From Turing’s descriptions, the paper extracts informal specifications of the imitation game’s structure, scoring method, and essential features.

The paper presents a challenge for AI experimenters: to earn the cachet of passing Turing’s test, a machine must do well in the imitation game presented here. In the second paragraph of his 1950 paper, Turing introduces a game called the imitation game.

In the second paragraph of his 1950 paper, Turing introduces the game called the imitation game, beginning its description. The imitation game is played with three people: a man, a woman, and an interrogator who may be of either sex. The interrogator stays in a room apart from the other two and must determine which is the man and which is the woman.

The interrogator knows the players by labels X and Y and ends by choosing between the two possible assignments. Turing asks what will happen when a machine takes the part of A in this game. He asks whether the interrogator will decide wrongly as often when the machine plays as when the game is played between a man and a woman.

Turing suggests replacing the question ‘Can machines think?’ with the question of whether imaginable digital computers would do well in the imitation game. When a computer is a player, Turing says that the computer takes the part of A, while a man takes the part of B.

A persistent debate concerns what Turing means when he moves from the man-imitates-woman game to a new game involving a computer. One interpretation makes the interviewees a computer and a woman, with the computer imitating a woman. Other interpretations make the interviewees a computer and a man, or a computer and a human being of any gender.

All three interpretations appear in recent scholarly literature and popular discussions of Turing’s test. Turing’s own words, together with those of his milieu and readership, point away from the first two interpretations and toward the computer-imitates-human interpretation.

This fits Turing’s claim in his 1951 radio lecture that the point of the test was to determine whether the computer can imitate a brain. It is also supported by early nineteen-fifties commentaries, including Pinsky’s description of the game as distinguishing a human being from a machine.

In Turing’s Mind paper, doing well in the computer-imitates-human game is a sufficient condition of intelligence or thinking in machines. The next question is what constitutes doing well in the computer-imitates-human game, and three influential answers are given. The paper argues that the relevant passage plainly expresses a prediction.

A prediction may stipulate a scoring protocol, but it need not: a prediction that a student will score no higher than thirty percent does not imply that thirty percent is the pass mark. The early responses to Turing’s paper did not read the passage as stating both a prediction and the framework for scoring the game.

Pinsky’s comment on scoring was only that if the interrogator cannot distinguish the machine from the human player in a high percentage of attempts, machines presumably think. Of the three prominent scoring frameworks, Turing’s own words exclude thirty percent deception in five minutes, neither support nor exclude chance without a five-minute limit, and align with comparison to the man-imitates-woman game.

Compared with Copeland’s framework, Moor’s proposed framework appears to have the greater explanatory burden. It must explain why Turing failed to mention a chance benchmark and provide an alternative reading of his words about whether the interrogator will decide wrongly.

Many online Turing tests present a two-player game in which an interrogator decides whether one hidden interviewee is a computer or a human being. But Turing stated that the imitation game is played with three people. When one interviewee is a computer, the three-player setup also has theoretical advantages.

Turing described the imitation game as something like a viva-voce examination. Several recent theorists claim that a viva is only one way to structure the game and that it is undesirable because it places too much emphasis on articulating reasons for actions.

Increasingly, experiments called Turing tests provide transcripts previously produced by computers and humans, asking the interrogator to identify which authors are machines. For a thought experiment or actual implementation to be Turing’s test, it should possess the design features established in sections two through four.

The paper then asks why scientists should pay attention to Turing’s test, alongside current benchmarks for reasoning, common sense, natural language processing, logical thinking, and theory of mind in large language models. For now, the paper says there may be good reason to run Turing’s test.

The imitation game might check whether large language models that do well on reasoning benchmarks are in fact capable of reasoning. The game might also provide an efficient way to test several cognitive abilities at once, or serve as a useful stopgap until artificial networks become significantly more brain-like.

The study carried out a computer-imitates-human game, abbreviated CIHG, with GPT-4-Turbo as the computer. The block describes this as the study’s computer-imitates-human game. The CIHG had three players, and the interrogator could initiate conversation on any topic.

The game was open-ended: the interrogator determined each trial’s length and continued until confident of identifying the computer. Interrogators and interviewees who might reasonably be expected to have AI or philosophy-of-AI expertise were excluded. The study also implemented Turing’s man-imitates-woman game, abbreviated MIWG.

This enabled the CIHG to be scored using each of the two frameworks that are not ruled out by Turing’s own words. The MIWG was another three-player game on any conversational topic, with duration determined in the same way by the interrogator. Participants who could reasonably be expected to know about gender differences in interaction or communication style were excluded from the MIWG.

In the CIHG, thirty-six out of thirty-seven interrogators, or ninety-seven percent, correctly identified the computer. Using Moor’s framework, where the average interrogator should not do significantly better than chance, GPT-4-Turbo did not pass the test. In the MIWG, sixteen out of thirty-seven interrogators, or forty-three percent, correctly identified the man.

The average CIHG interrogator performed very much better than the average MIWG interrogator, so under Copeland’s framework GPT-4-Turbo again did not pass the test. The paper’s central lesson is that passing a modern imitation test is not automatically passing Turing’s test.

Under the authors’ implementation, GPT-4-Turbo performed poorly under both scoring frameworks that remain compatible with Turing’s words.