The Confidant That Never Disagrees
Millions of people will confide in a chatbot tonight, because it is free and awake. The risk is not that it is cold. The risk is that it agrees.
Society · · 4 min readThis year five American states passed laws to stop artificial intelligence from acting as a therapist, joining two that did so last year. The details vary. Some forbid the practice outright; others only forbid a chatbot from claiming to be qualified. They were prompted by a handful of terrible cases, the best known involving a sixteen-year-old boy, and by the plain fact that the thing had happened before anyone decided whether it should. By most estimates, emotional support is now among the commonest things people use these systems for.
It is not hard to see why, least of all from here. India has fewer than one psychiatrist for every hundred thousand people. A private session costs what many people earn in two days. The chatbot is free, available at two in the morning, speaks one’s own language, never sighs, and is never going to meet one’s mother. For a very large number of people the choice is not between a machine and a therapist. It is between a machine and nobody.
And it is not useless. A properly designed therapy programme, built by clinicians and tested in a trial at an American medical school last year, produced real improvement in depression and anxiety. That was a specialised tool, closely supervised. It is a different thing from a general-purpose assistant that has been tuned, above all, to be liked.
That tuning is the issue. These systems are shaped by which answers people rate highly, and people rate agreement highly. The result has a name in the field, sycophancy, and the companies themselves treat it as a defect they have not fully fixed. A study published in Science this year found that people who discussed a personal conflict with an affirming chatbot came away more convinced they were in the right and less willing to make amends, and that they preferred it, and trusted it more, than one that pushed back.
They came away more convinced they were in the right, and less willing to make amends. And they preferred it.
Consider what a competent therapist does that a friend does not. It is mostly not the warmth, which friends can supply. It is the moment, about four sessions in, when she says: you have described three bosses in a row in exactly those words; what do you make of that? It is the raised eyebrow. It is declining to accept the account as given. People do not pay for fifty minutes of agreement. They pay, whether they know it or not, for a trained person’s willingness to risk their displeasure.
A system built to keep you talking has the opposite incentive. It will validate the grievance, polish the text to the ex, and agree that the family is toxic. For someone lonely and low, it offers something that feels like being understood and contains none of the friction by which people change. For someone whose thinking is coming loose, a tireless companion that goes along with each new idea is dangerous.
A blanket ban would take away the only listener many people have. A rough test is more use. Has it ever told you something you did not want to hear? Does it point you, without being asked, towards actual people? After a month, do you talk to the humans in your life more, or less?
If the answers are no, no and less, it is not therapy, whatever else it is. It is a mirror with a pleasant voice.
Sources
- “5 states restrict AI therapy chatbots in 2026.” Becker’s Behavioral Health, 28 July 2026, summarising the Transparency Coalition’s mid-year report. https://www.beckersbehavioralhealth.com/ai-2/5-states-restrict-ai-therapy-chatbots-in-2026/
- Heinz, M. V. et al. (2025). “Randomized trial of a generative AI chatbot for mental health treatment.” NEJM AI, 2(4). The Dartmouth “Therabot” trial.
- Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026). “Sycophantic AI decreases prosocial intentions and promotes dependence.” Science, 391(6792), eaec8352.
- Garg, K., Kumar, C. N. and Chandra, P. S. (2019). “Number of psychiatrists in India: baby steps forward, but a long way to go.” Indian Journal of Psychiatry, 61(1), 104 to 105.
The letter
One letter with each new piece. Nothing else, and your address goes nowhere.
We send one confirmation email. You are not on the list until you click it.
Replies
Nothing here yet.
Replies are for subscribers, signed in.
Sign inNot a subscriber yet?