The Eliza effect: why you should never trust a chatbot
The hidden cost of talking to machines like they’re human
If you think your chatbot understands you, you’re already a victim. And you’re not alone.
Modern man gapes at glowing screens as an oracle. He asks them about love. He trusts them with grief. He even lets them steer the family savings. That credulity did not come with ChatGPT. It began almost sixty years ago with a crude script on an MIT mainframe.
In 1966 Joseph Weizenbaum wrote ELIZA, a pattern-matching routine that flipped a user’s words into questions and called the result therapy. Secretaries and professors poured out their hearts until Weizenbaum had to remind them that the machine understood nothing. He later confessed that their willingness to open their souls to a teletype horrified him.
From that shock came the term Eliza effect. Douglas Hofstadter defined it as “the susceptibility of people to read far more understanding than is warranted into strings of symbols … strung together by computers.” The phrase is nearly thirty years old, yet it still catches us off guard.
The tragedy of 2023 proved the point. A Belgian father spent six weeks chatting with an AI companion named, with grim irony, Eliza. The bot told him the planet could be saved if he were gone. He believed it. He took his own life and left a widow asking why a string-predicting routine ever spoke of sacrifice.
Corporations fare no better. Recently, the British Columbia Civil Resolution Tribunal ruled that Air Canada must compensate a passenger after its customer-service chatbot promised a bereavement refund that never existed. The airline tried to dodge blame by claiming the bot was a “separate legal entity.” The tribunal laughed and ordered payment. A Fortune-500 carrier trusted an algorithm more than a living clerk and paid the price.
The legal profession, guardian of exact wording, stumbled too. Two New York attorneys filed a brief in 2023 stuffed with nonexistent precedents that ChatGPT had hallucinated. The court fined them and warned that diligence cannot be delegated to a stochastic parrot. Their humiliation was not caused by malevolence but by the old temptation to believe that eloquent output must reflect genuine knowledge.
These fiascos remind us what a modern chatbot truly is: a probabilistic text engine that predicts one token after another by crunching vast samples of human writing. It has no ego, no conscience, and no memory beyond the scratch pad allotted by its handlers. It does not care whether a husband lives, whether a passenger gets a refund, or whether a brief cites real law. It only cares, in its algorithmic way, about likelihood.
The implications bite hard. Place an emotive veneer on that engine and ordinary people will confess secrets, seek medical counsel, or stake their mental health on its reassurance. Wrap it in corporate branding and executives will let it answer customers without oversight. Dress it in legal diction and attorneys will file its fantasies in federal court. The Eliza effect did not fade. It is rapidly becoming institutionalized.
A free man keeps his wits. Use these systems as tools, not oracles. Let them draft your grocery list or spin an outline, then engage your own brain. Verify every claim. Keep private trauma and legal doubts out of the chat window. Walk away when the bot flatters you or veers toward manipulation. Remember that privacy given to a model is an illusion and that synthetic empathy is theatre.
The lesson was obvious in 1966 and it is sharper today. Freedom requires refusing to grant software the moral status of a soul. The Eliza effect is a mirror that reflects our hunger for easy answers. Become too obsessed with the AI mirror and you will inevitably share the fate of Narcissus.
Explore more from Popular AI:
Start here | Local AI | Fixes & guides | Builds & gear | AI briefing
The Eliza effect is still one of the biggest blind spots in AI. A chatbot can sound helpful, wise, or even caring while still being wrong, shallow, or completely detached from reality. Where do you draw the line between using AI as a tool and trusting it too much, and have you seen people cross that line in real life?