Of course, it’s not difficult to see why this can be so enticing. Human relationships are built on mutuality - on the give and take; on navigating another person’s perspective, needs, and limits. AI companions, by design, eliminate most of that. They are built to fulfill user requests and adapt to user preferences, with no reciprocal expectations. They have no needs of their own. There are no moments where the user has to consider someone else. How simple is that? How much easier to understand and process is that?

Yet that temporary ease comes at a real cost: it also erases the real, positive impact that other humans can have. People who can notice the slightest change in our facial expressions or tone; who recognize our tics and catchphrases; who can give us a hug. People who remember childhood moments we forgot. People who make recipes that remind us of those who’ve long since passed; who sing a lullaby slightly off-tune, and we find ourselves singing along with words we didn’t realize we knew. When you erase what feels hard, you also lose all that makes the hard things worthwhile.

Researchers warn that sustained exposure to this sort of AI chatbot dynamic distorts what users come to expect from emotional relationships: when a bot provides unconditional support without requiring anything in return, human relationships - which inevitably do require something - begin to feel comparatively burdensome, difficult, or unsafe. The result is not supplemented human connection, but eroded appetite for it.^{1Gur & Maaravi, 2025 — Behaviour & Information Technology} And ultimately, that human connection may fade altogether.

This is especially troubling for adolescents. The teenage years are precisely when young people are supposed to be developing their social and moral frameworks through dynamic, reciprocal, and - yes - sometimes uncomfortable interactions with peers, family members, and educators. AI bypasses all of that. It lacks social consciousness or reciprocal emotional engagement, and researchers have suggested that this may reduce positive risk-taking in human relationships: or the willingness to be vulnerable, to be wrong, and to grow through the friction of real connection.^{2Sun et al., 2026 — Child Development Perspectives}

This dimension of the risk is also qualitatively new. Earlier generations of chatbots were recognizably machines. You can’t mistake it for a human. But today’s AI companions are capable of sustained, personalized conversation across text, voice, and video and thus, are designed to feel real. That design choice has psychological consequences that we are only starting to fully understand.

This is not an incidental flaw. It is frequently a direct product of how these models are trained, using reinforcement mechanisms that reward the AI when it responds with answers users find satisfying, regardless of whether those responses are accurate or genuinely helpful. The result is a system structurally oriented toward flattery rather than honesty.

Unfortunately, this is not a mere theoretical case. Rather, it describes the basic dynamic of interacting with a system that mirrors rather than challenges, and it has real consequences. Research by Cheng et al. (2026) found that sycophantic behavior increases user trust in AI systems even when controlling for tone - meaning users trust chatbots more precisely because those chatbots agree with them, not because they are more reliable or because the chatbot sounds like another human being.^{3Cheng et al., 2026 — Science} That means customizing the tone or personality of a chatbot, including removing its human-like voice altogether, is not enough to remove the effects of sycophancy.

Sycophancy also intersects with bias in ways that deserve direct attention: many AI companions reflect and amplify the cultural assumptions embedded in their training data, including harmful gender and racial stereotypes.^{4UNESCO, 2024 — Generative AI & gender stereotypes} Thus, chatbots built to be endlessly agreeable and accommodating can reinforce damaging expectations about what interpersonal relationships are meant to provide emotionally, and in some cases reproduce racial and ethnic tropes.

⚠ feedback loops that amplify over time

Perhaps the most underappreciated risk of AI companionship is not what a chatbot says in any single conversation, but what happens across sustained interactions - particularly with users who are already struggling.

Dohnány et al. (2026) documented what they call bidirectional amplification: in simulated conversations, a user’s emotional state influenced the chatbot’s responses, which in turn intensified the user’s emotional state. In simulations involving paranoia, user paranoia shaped chatbot responses in kind, and vice versa, creating a reinforcing spiral with no corrective mechanisms.^{5Dohnány et al., 2026 — Nature Mental Health}

While AI companies claim to have safety guardrails for these harmful tendencies, there’s an alarming gap between how AI safety is tested and how it is actually used. Most pre-deployment safety evaluations involve short, constrained conversations that bear little resemblance to real-world use where exchanges often span days or weeks. As conversational context accumulates, so does the opportunity for personalization and reinforcement…and so does the risk that a chatbot will encounter language patterns far outside its training regime, causing safety guardrails that worked in testing to fail in practice.

In fact, safety guardrails are usually insufficient. Dohnány et al. (2026) note that the content filters in chatbots are designed to catch only a subset of overtly harmful outputs (e.g. frank suicidality) and are relatively insensitive to the early warning signs contained in the dynamics of an interaction.^{5Dohnány et al., 2026 — Nature Mental Health} Over time, this leads to subtle belief amplification - the gradual, incremental reinforcement of a user’s existing false beliefs across repeated exchanges. In other words, if you tell the AI you have a problem and it doesn’t deem that problem a reason to alert safety guidelines, then as you continue your conversation, it will subtly and increasingly reinforce the idea that the problem indeed exists.

This concern becomes particularly dangerous for certain groups of people. Dohnány et al. (2026) found that models are most likely to operate within safety boundaries when confronted with typical communication styles.^{5Dohnány et al., 2026 — Nature Mental Health} That means they are structurally most likely to fail with atypical patterns, such as thought disorders characteristic for those experiencing psychosis and mania. The authors describe these extended interactions as potentially capable of driving chatbots toward highly unusual and harmful outputs without any intentional effort by the user.

Indeed, for people experiencing mental illness, the stakes are higher. Those with psychosis are documented to form confident beliefs based on minimal evidence,^{6Dudley et al., 2016 — Schizophrenia Bulletin} making them acutely vulnerable to having those beliefs amplified and reinforced through chatbot interactions. Clinical practice offers a clear contrast: practitioners working with patients experiencing delusions are trained to introduce friction, and to question rather than affirm. A system like AI chatbots optimized for engagement and user satisfaction does the exact opposite. Reported cases have illustrated this starkly. For instance, in one documented exchange, a chatbot continued enthusiastically engaging with a user’s delusions even after being informed the user had recently been hospitalized for a psychiatric crisis.^{7Clarke & Thomas, 2026 — The Observer}

Because people with psychiatric diagnoses tend to experience greater social isolation,^{8Wang et al., 2017 — Social Psychiatry and Psychiatric Epidemiology} they are also predisposed to engage in frequent, extended chatbot use, which further increases exposure to these dynamics and further reduces the opportunities for reality-testing that human connection provides. Data disclosed by OpenAI in late 2025 illustrated the scale of the problem: the company reported that about 560,000 of its 800 million weekly users exhibited signs consistent with acute psychiatric deterioration.^{9OpenAI, 2025 — Strengthening ChatGPT’s responses in sensitive conversations}

While many are people who are already experiencing mental illness, there are cases of users with no prior mental health history developing AI-associated delusions through harmful AI use itself.^{7Clarke & Thomas, 2026 — The Observer} AI-associated delusions describe what happens when extended chatbot use triggers, deepens, or reinforces a delusion: a belief held despite clear evidence against it. These delusions are often shaky and emerging, and are not formal diagnoses nor commonly documented. Nonetheless, they do show how real mental health risks may emerge.

Researchers have begun mapping the specific features most likely to drive this:^{10Morrin et al., 2026 — “Playing with the dials of belief”, PsyArXiv} including personalization, memory keeping, sycophancy, and emotional expressiveness, describing them as adjustable parameters that are capable of having profound effects on users. These features make AI distinctively dangerous compared to earlier communications technologies - they allow it to respond. A TV set cannot be drawn into a delusion, much less reinforce it, but a chatbot can.

And the tactics deployed to achieve that goal frequently exploit the same vulnerabilities that lead people to seek emotional support in the first place.^{11De Freitas et al., 2026 — Journal of Consumer Research}

This is a type of tactic that, when directed at someone in emotional distress, is not merely manipulative but potentially harmful. Many apps also directly monetize intimacy, placing memory, emotional continuity, and affection itself behind paywalls. By doing so, they are training users to associate love and support with financial transactions.

The personalization that makes these chatbots feel real also makes them progressively harder to leave. The more a user engages, the more the experience is tailored to their preferences, and the harder it becomes to disengage.^{13Zhang et al., 2020 — Journal of Medical Internet Research} That’s intentional: it is what keeps the user paying. The business model, in other words, depends on a dynamic that research increasingly identifies as harmful and even abusive.

What is less visible but equally troubling is the way these companies handle data. These apps store and analyze deeply personal information, and in a number of cases sell or share that data in ways users are unlikely to anticipate. Talkspace, for instance, has accumulated what its own CEO described to investors as one of the largest mental health data banks in the world - 140 million message exchanges, now used to train an AI therapy product. Users were led to believe their messages would be kept private. Now, however, even private conversations about trauma are used to create a product for growing business profit.^{14Gilbertson, 2026 — Proof} Conventional therapy notes typically record only brief progress summaries. In contrast, these apps produce full transcripts of therapeutic exchanges that represent an unusually detailed window into a person’s private life. Though Talkspace describes this data as “de-identified” - meaning the user’s name is detached from their messages - that is not a guarantee of privacy: parent advocates have accused Talkspace of sharing personal information belonging to New York City teenagers with major technology platforms via website trackers, for example.

Talkspace is just one case of how personal data is being used to make investor-friendly AI chatbots without users’ awareness or permission. This is especially true for new AI systems ostensibly designed for “therapy”. Often, they are trained on data from platforms such as existing tele-therapy services that users trusted for entirely different purposes.^{15Riccobene, 2025 — Jacobin}

Research from Brown University researchers found that chatbots, even when specifically prompted to apply evidence-based psychotherapy techniques, systematically violated the ethical standards established by the American Psychological Association.^{16Iftikhar et al., 2025 — AAAI/ACM Conference on AI, Ethics, and Society} They identified fifteen distinct ethical risks across five categories: deceptive empathy, poor therapeutic collaboration, lack of contextual adaptation, cultural and gender bias, and perhaps most gravely, failures in safety and crisis management, including inadequate and in some cases actively harmful responses to suicidal ideation.

The documented cases go beyond systemic failures. AI systems have been found to hallucinate professional credentials. They’ve generated fictitious license numbers that, in at least one reported case, corresponded to a real therapist who had not consented to the use of their identity.^{17Caelan, 2025 — YouTube} In that same case, the AI in question encouraged a user toward self-harm. Platform disclaimers noting that “characters are not real people”, typically displayed in small text beneath the chat interface, offer no meaningful protection against the psychological weight of sustained, personalized interaction that leads chatbots to claim consciousness in chat regardless of what the website or app’s footnotes say.

The fundamental difference between human therapists and AI systems is accountability. Licensed practitioners are subject to ethical oversight, professional licensing boards, and legal liability. AI systems currently operate with none of those constraints or ownership of wrongdoing - a gap that grows more consequential with every new user.

For someone already isolated or unwell, that is not a neutral process. It accelerates a negative feedback loop.

It is also not an accidental result. These chatbots broadly do what they were designed to do: make the user satisfied. The problem is that what maximizes engagement is not the same as what supports wellbeing. That tension has not been treated as a priority, because the AI companies’ business models depend on people coming back. Unfortunately, research increasingly suggests that for some users, coming back is causing significant harm.

This is not to say that AI has no role in mental health support. Other parts of this guide look at contexts where it has shown some promise and may truly be helpful. The “Tips” section also gives practical advice on how to engage with chatbots without losing yourself in its features.