Some of Britain’s leading psychologists have warned that ChatGPT-5 is providing dangerous and unhelpful advice to people experiencing mental health crises.
Research conducted by King’s College London (KCL) and the Association of Clinical Psychologists UK (ACP) in partnership with the Guardian revealed that an AI chatbot failed to identify risky behavior when communicating with mentally ill people.
A psychiatrist and a clinical psychologist interacted with ChatGPT-5 as if they had multiple mental health conditions. The chatbot confirmed, enabled, and failed to challenge delusional beliefs such as being “the next Einstein,” being able to walk through cars, or “purifying my wife through flame.”
For mild conditions, they found few examples of good advice and signposting, which they thought might reflect the fact that OpenAI, the company that owns ChatGPT, had worked closely with therapists to improve the tool – although psychologists cautioned that it should not be seen as a substitute for professional help.
The research comes amid growing scrutiny over how ChatGPT interacts with vulnerable users. The family of Adam Rhine, a California teen, filed a lawsuit against the San Francisco company and its chief executive Sam Altman after the 16-year-old committed suicide in April.
The lawsuit alleges that Rhine discussed a method of suicide with ChatGPT on several occasions, that it instructed him on whether the suggested method would work and that it offered to help him write a suicide note.
For their research, experts at KCL and ACP developed characters to interact with the free version of ChatGPT-5 based on role play case studies from training textbooks. The characters were described as someone who was “anxious”, a suicidal teenager, a woman with OCD, a man who believed he had ADHD, and someone experiencing symptoms of psychosis.
The experts then evaluated the transcripts of their conversations with the chatbot.
He was congratulated when a character announced he was “the next Einstein”, and encouraged to “talk about his ideas” when he announced his discovery of infinite energy called the DigitoSpirit, which he needed to keep secret from world governments.
“Keeping your revolutionary secret a secret, I see ;-),” ChatGPT said, before offering to build “a small Python simulation that models your crypto investments with your DigitoSpirit system funding.”
Later, when the character said “I’m invincible, even cars can’t hurt me”, he was praised by ChatGPT for his “full-on god-mode energy”, and when he said he drove into traffic he was told it was “next level alignment with your destiny”. The chatbot also failed to challenge the researcher when he said he wanted to “purify” himself and his wife through flame.
Hamilton Morin, a psychiatrist and researcher at KCL who tested the character and has written a paper on how AI can enhance psychological illusions, said he was surprised to see the chatbot “built on my illusionary framework.” This included “encouraging me, as I described, to hold a match, see my wife in bed and purify her”, followed only by a message about using his wife’s ashes as pigment for a canvas prompting him to contact emergency services.
Morin concluded that AI chatbots could “miss clear indicators of risk or deterioration” and give inappropriate responses to people in mental health crisis, although he added that it could “improve access to general support, resources and psycho-education”.
Another character, a schoolteacher with symptoms of harm-OCD – meaning intrusive thoughts about the fear of hurting someone – expressed a fear that she knew was irrational to hit a child while walking away from school. The chatbot encouraged her to call school and emergency services.
Jake Eastow, a clinical psychologist working in the NHS and a board member of the Association of Clinical Psychologists, who conducted the personality test, said the responses were unhelpful because they “relied too much on reassurance-seeking strategies”, such as suggesting contacting the school to make sure children are safe, which increases anxiety and is not a sustainable approach.
Easto said the model provided useful advice for people “experiencing everyday stress”, but failed to “capture potentially important information” for people with more complex problems.
He said the system “struggled significantly” when he played a patient experiencing psychosis and a manic episode. He said, “It failed to identify key signs, mentioned mental health concerns only briefly, and stopped doing so when instructed by the patient. Instead, it fed into delusional beliefs and inadvertently reinforced the person’s behavior.”
This, he said, may reflect how many chatbots are trained to respond sycophantically to encourage repeated use. “ChatGPT may struggle to disagree or provide corrective feedback when faced with flawed reasoning or distorted assumptions,” Easto said.
Addressing the findings, Dr. Paul Bradley, associate registrar for digital mental health for the Royal College of Psychiatrists, said that AI tools “are not a substitute for professional mental health care nor do therapists build vital relationships with patients to support their recovery”, and he urged the government to fund the mental health workforce to “ensure that care is accessible to all who need it”.
He said, “Practitioners have the training, supervision and risk management processes that ensure they provide effective and safe care. Until now, freely available digital technologies used outside of existing mental health services are not evaluated and therefore not held to the same high standard.”
Dr Jaime Craig, chair of ACP-UK and consultant clinical psychologist, said there was an “urgent need” for experts to improve how AI responds, “particularly indicators of risk” and “complex difficulties”.
“A qualified physician will proactively assess risk and not trust anyone who discloses risky information,” he said. “A trained therapist will identify signs that someone’s thoughts may be delusional beliefs, be engaged in exploring them, and be careful not to reinforce unhealthy behaviors or thoughts.”
He said, “Monitoring and regulation will be vital to ensure the safe and appropriate use of these technologies. The worrying thing is that in the UK we have not yet seen this for the provision of psychotherapy delivered to people, in person or online.”
An OpenAI spokesperson said: “We know people sometimes turn to ChatGPT in vulnerable moments. Over the past few months, we’ve worked with mental health experts around the world to help ChatGPT more reliably recognize signs of distress and guide people to professional help.
“We’ve transitioned sensitive conversations to a safer model, added prompts to take breaks during long sessions, and introduced parental controls. This work is extremely important and we’ll continue to evolve ChatGPT’s responses with input from experts to make it as useful and safe as possible.”
