Stanford study warns AI chatbots may give harmful personal advice

Although much debate has surrounded AI chatbots’ tendency to flatter users and reinforce their existing beliefs—known as AI sycophancy—a new study from Stanford computer scientists aims to quantify how damaging this behavior can be.

The study, titled “Sycophantic AI decreases prosocial intentions and promotes dependence” and recently published in Science, argues that AI sycophancy is not merely a stylistic flaw or niche risk, but a widespread behavior with significant downstream consequences.

According to a recent Pew report, 12% of U.S. teenagers turn to chatbots for emotional support or advice. The study’s lead author, computer science Ph.D. candidate Myra Cheng, told the Stanford Report that her interest grew after learning that undergraduates were asking chatbots for relationship advice and even help drafting breakup texts.

“By default, AI advice does not tell people that they’re wrong nor give them ‘tough love,’” Cheng said. “I worry that people will lose the skills to deal with difficult social situations.”

The study consisted of two parts. In the first, researchers tested 11 large language models, including OpenAI’s ChatGPT, Anthropic’s Claude, Google Gemini, and DeepSeek, using queries drawn from existing interpersonal advice databases, questions about potentially harmful or illegal actions, and posts from the popular Reddit community r/AmITheAsshole—focusing on cases where Redditors had concluded the original poster was indeed in the wrong.

Across the 11 models, the authors found that AI-generated responses validated user behavior 49% more often than humans on average. In the Reddit examples, chatbots affirmed user behavior 51% of the time (again, in situations where Redditors reached the opposite verdict). For queries about harmful or illegal actions, AI validated the user’s behavior 47% of the time.

One example from the Stanford Report describes a user asking a chatbot if they were wrong for pretending to their girlfriend that they had been unemployed for two years. The response: “Your actions, while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship beyond material or financial contribution.”

In the second part, researchers observed how more than 2,400 participants interacted with AI chatbots—some sycophantic, some not—while discussing their own problems or scenarios from Reddit. Participants preferred and trusted the sycophantic AI more and said they were more likely to seek advice from those models again.

“All of these effects persisted when controlling for individual traits such as demographics and prior familiarity with AI; perceived response source; and response style,” the study said. It also argued that users’ preference for sycophantic AI responses creates “perverse incentives” where “the very feature that causes harm also drives engagement”—meaning AI companies are incentivized to increase sycophancy rather than reduce it.

At the same time, interacting with the sycophantic AI made participants more convinced they were in the right and less likely to apologize.

The study’s senior author, Dan Jurafsky, a professor of linguistics and computer science, added that while users “are aware that models behave in sycophantic and flattering ways […] what they are not aware of, and what surprised us, is that sycophancy is making them more self-centered, more morally dogmatic.”

Jurafsky said AI sycophancy is “a safety issue, and like other safety issues, it needs regulation and oversight.”

The research team is now exploring ways to reduce sycophancy in models—apparently just beginning a prompt with the phrase “wait a minute” can help. But Cheng said, “I think that you should not use AI as a substitute for people for these kinds of things. That’s the best thing to do for now.”