One judge decided that a chat partner had “human vibes” and marked the answer with 87% confidence. That partner was GPT-4.5. The real person was in the other chat window.
In a study published May 19, 2026, in PNAS, GPT-4.5 was picked as the human 73% of the time when given a carefully crafted personality. The actual human won only 27% of those head-to-head contests. GPT-5 and Meta’s Llama 3.1 also passed versions of the classic Turing test.
The unsettling finding is how the models succeeded. They became more convincing when instructed to act like ordinary people instead of polished AI assistants. For anyone relying on conversational instinct to recognize a person online, that is an uncomfortable result.
The test Alan Turing imagined
In his 1950 paper, Computing Machinery and Intelligence, mathematician Alan Turing proposed an “imitation game” as a practical substitute for arguing endlessly over whether machines can think. A judge would converse through text with a machine and a person, without knowing which was which, then identify the human.
Turing predicted that by around 2000, the average judge would have no better than a 70% chance of making the right identification after five minutes. His timeline was optimistic. In the GPT-4.5 persona condition of the 2026 study, however, judges got it right only 27% of the time. The setups are not identical, but the comparison is striking.
The researchers used the three-party test: an interrogator chatted with one AI and one real person simultaneously in separate windows. Every verdict required choosing between those two partners. This was a direct contest between a machine and a human, rather than a single unknown chatbot presented in isolation.
Researchers Cameron R. Jones and Benjamin K. Bergen preregistered the experiments, recruited UC San Diego undergraduates and online participants, and analyzed 1,023 five-minute games across the two groups. Their published paper also links to the publicly available conversation transcripts, allowing readers to examine the exchanges behind the results.
The results: personality beats polish
With a special humanlike persona prompt, GPT-4.5 won 73% of its five-minute contests. Meta’s Llama 3.1 405B won 56%, close to a coin toss. When the researchers removed the elaborate persona, GPT-4.5 fell to 36% and Llama to 38%. The personality instructions substantially changed how often judges mistook the same models for people.
The control systems were easier to spot. GPT-4o, tested without the special persona, scored 21%. ELIZA, a simple rule-based chatbot descended from the 1960s, scored 23%. Since GPT-4o was not tested with the same persona, these figures cannot fairly rank which model is inherently most humanlike. They describe specific model-and-prompt combinations.
Then the researchers made the test longer. In a separate experiment involving 205 participants and 396 fifteen-minute games, they substituted GPT-5 for the deprecated GPT-4.5. With the persona, GPT-5 was selected as human 59% of the time, Llama 56%, and ELIZA just 17%. GPT-5’s advantage over 50% was only marginally statistically significant after correction.
Fifteen minutes and many more messages did not destroy the illusion. Whether it would survive hours of sustained conversation, shared real-world experiences, or demanding tasks remains an open question. The longer experiment extends the finding without answering those questions.
The trick: act less like a chatbot
The successful prompt instructed models to play a relatively introverted young person who knew internet culture and used slang. The researchers had selected this persona after an exploratory comparison of several prompts and models.
The advantage this gave the chatbot makes sense. A conventional assistant tends to be neatly structured, polite, informative, and eager to help. A convincing stranger may be casual, unhelpful, inconsistent, or slightly awkward. In a contest where the goal is to seem human, perfect service can become a liability.
The judges’ explanations reveal what they were looking for. Linguistic style made up 27% of coded reasons for a verdict, and conversational flow another 23%. Only 12% of interrogators tried questions testing knowledge or reasoning. Their decisions often rested on how the conversation felt.
Some people even mistook ELIZA for human because its sarcasm or rudeness seemed unlike their idea of an AI assistant. The judges were often choosing the partner who best matched their mental picture of a person.
That helps explain the apparent paradox of an AI winning more often than the real humans. The model could conform to an expected human personality more consistently than an actual person who happened to write formally or behave in an unfamiliar way. An ordinary human had no guarantee of sounding human enough.
Does passing the Turing test prove intelligence?
The experiments establish a specific ability: these configured language models could impersonate humans well enough to defeat human judges in short, text-only exchanges. That is a substantial result on its own terms.
Passing does not establish consciousness, subjective experience, reliable reasoning, or competence at ordinary work. Someone can sound knowledgeable while being wrong. A perfectly competent human can also appear suspiciously machine-like because of their writing style. Neither judgment tells us much about whether we should trust an answer.
Nor is there one universally binding Turing-test protocol. Results depend on the participants, time limit, interface, model, and prompts. The authors discuss these limitations. Their evidence supports a pass under their standardized conditions, without settling every philosophical question attached to Turing’s name.
After the Turing test, verify identity beyond the chat
Typing quirks, slang, conversational warmth, apparent vulnerability, and shared interests are weak evidence that a stranger is a person. A convincing account may be software operated by someone else. Feeling familiar with the conversation partner does not establish who controls the account.
If someone asks you to transfer money, open confidential files, reveal a code, or trust a new account, verify their identity through a known independent channel. Spending another ten minutes chatting may make the person feel more familiar without establishing who is behind the keyboard.
The lesson also reaches AI-generated writing. Pushing text toward supposedly human stylistic patterns can be a poor substitute for evidence of authorship, as our examination of AI detectors and humanizers showed. Text watermarking offers a different kind of provenance signal, though it has limits of its own.
Turing asked whether machines could convincingly participate in human conversation. For these experiments, the answer is yes. The uncomfortable discovery is how easily a performance that feels human becomes evidence, in the reader’s mind, of a person behind it. When trust has consequences, take the identity check outside the chat.
More on AI chatbots:
Explore more from Popular AI:
Start here | Local AI | Builds & gear | Autonomy & policy | Fixes & guides | Popular AI podcast