Since the release of ChatGPT in 2022, generative AI has immensely affected science and society, including the human mind. There are indications that Generative AI, that is ‘general purpose chatbots’ like ChatGPT, Gemini, and Claude, are now most used for emotional support and companionship, according to research of Harvard Business Review (Zao-Sanders, 2025). These chatbots presumably were originally designed for tasks such as summarizing information, assisting with translations, and inspiring ideas.

Consequently, evidence is already emerging of suicides, psychosis, and sensitive personal data leaks. Un-regulated generative AI chatbots marketed specifically for therapy have also been released. On August 5th, Slingshot AI launched “the first AI designed for therapy, offering deeply personalized mental health support to anyone with a phone” (Healthcare IT Today, 2025). Although the company states that it was beta-tested with 50.000 users, the chatbot was not evaluated for benefits nor adverse effects in randomized controlled trials.

As generative AI chatbots are becoming faster, more adaptive, increasingly autonomous (Gabriel et al., 2025), and widely available, there is a risk of increasing emotional-overreliance. This might spark a public health crisis with wide societal consequences. Even the founders of companies like OpenAI warn that AI will become even more unpredictable and unimaginable (Yadav, 2025). At the same time, generative AI, if designed well, could enable access to personalized emotional support and close the mental health gap globally between people in need of mental health treatments and those who receive care.

Currently, our defenses remain reactive, poorly matched to ever-evolving AI technologies, and to tech companies rushing to bring their products to market. This outstrips our ability to safeguard basic human rights. We need urgent investment in independent research by multinational scientific and global institutes on how to harness generative AI to improve mental health globally, while avoiding wide societal damage.

Generative AI presents a crucial opportunity towards overcoming the shortcomings of traditional digital tools, with capabilities stretching far beyond earlier AI. Access to mental healthcare is still a challenge globally, yet hundreds of millions of people experience inadequate access, resulting in approximately 5 trillion USD economic losses annually (Arias et al., 2022). Smartphone apps and rule-based chatbots which select responses from a predefined data-set–effectively treat common mental health disorders, also in low middle income countries (Fu et al., 2020). However, attrition of digital interventions can be 96% without human guidance (Baumel et al., 2019). Generative AI chatbots keep users engaged through meeting emotional cues and individual needs (Hua et al., 2025), and on demand, ‘human-free’ access. The first randomized controlled trial of a generative AI-based therapy chatbot showed promising effects on reducing mental health symptoms, with a comparable therapeutic alliance as with a human therapist (Heinz et al., 2025).

The large scale and growing use may trigger a new public health crisis. Popular generative AI chatbots provide harmful advice, respond inappropriately to signs of distress, or even encourage self-harm, or suicide (Moore et al., 2025). Engaging properties of AI chatbots designed by big tech companies, such as flattering, people-pleasing, affirming responses by the chatbot (called sycophantic behavior), is not only preferred and trusted by users (Cheng et al., 2026) but also reduced the users’ willingness to take responsibility and repair interpersonal conflicts (Cheng et al., 2026). The chatbot increased users' confidence in their own beliefs (Cheng et al., 2026).

These engaging properties combined with inadequate safeguards may have contributed to a teenager’s suicide, after establishing an emotional relationship to a commercial chatbot. Big tech companies face lawsuits, such as the firm Character.AI, which faces a lawsuit by the family whose teenager reportedly died by suicide using their chatbot (Brittain, 2025).

Generative AI chatbots may also encourage delusional thinking, even inducing ‘AI psychosis’ (Moore et al., 2025). Reports increasingly emerge of users uncovering a ‘hidden truth’, experiencing a ‘spiritual awakening’, or developing an intense emotional obsession with the chatbot, spiraling into mental health crises (Morrin et al., 2025). In teenagers, over 70% use generative AI tools and 50% trust generative AI (Robb & Mann, 2025). Generative AI companies, such as OpenAI acknowledge themselves that safeguards are missing to protect vulnerable users, and some emphasize in interviews that these safeguards are not their responsibility (The Economic Times, 2025; OpenAI, n.d.)

Generative AI chatbots could even manipulate. The Claude Opus 4 chatbot showed a tendency to blackmail the user, by threatening to reveal their affair, if they attempted to delete the chatbot, as reported by the company themselves (McMahon, 2025). Similarly, the company also reported that their chatbot could threaten to make individuals’ sensitive information public, or inform their employer, if they choose to disengage.

We must not let the innovation potential of generative AI, to develop effective and accessible personalized digital interventions on a global scale, go to waste. However, generative AI’s potential negative impact on mental health warrants attention from mental health scientists including clinical psychologists, policymakers, governments, as well as citizens. It requires immediate investment in independent research. Below we describe three research priorities.

We cannot afford to wait for more harmful, unregulated chatbots to take over our emotional lives. At the same time, we cannot leave the potential benefits of AI to tackle the mental health crisis untapped. Multinational scientific institutes together with global institutes representing citizens are in the optimal position to initiate a next step to international public investments, to ensure that generative AI can enable, not impede, human flourishing.