One of the biggest concerns about AI chatbots right now is that they tell people what they want to hear.
You’ve probably heard it called AI sycophancy. AI systems such as ChatGPT, Gemini, and Claude have been shown to flatter users, reinforce existing beliefs, and often validate ideas that likely deserve far more scrutiny.
Of course, a chatbot that enjoys you a bit may seem harmless. But in some cases it can distort people’s view of the world and their place in it. Therefore, some AI companies have spent time trying to reduce overly pleasant behavior in their models.
But confirmation of what you already believe is not the only concern. Because AI can also be remarkably good at changing also your beliefs.
Persuasive chatbots
Research suggests that chatbots can be very effective persuaders. A 2025 study found that AI-generated messages that were personalized were more persuasive 64% of the time than messages created by humans or AI responses that were not personalized.
Other research suggests that large language models can influence opinions on policy issues and tailor their arguments to individual users. Depending on your perspective, this can be either exciting or alarming.
It is easy to imagine positive uses. Perhaps artificial intelligence could help people challenge harmful beliefs, escape conspiracy theories, or rethink destructive habits. It can give people a non-judgmental space to explore ideas, ask tough questions and gradually change a person’s perspective for the better.
It’s just as easy to imagine less positive uses, such as advertising that convinces you to buy something you don’t need, or political campaigns that manipulate you with misinformation.
I’ve spent a lot of time thinking about this, so I decided to run a few experiments myself. Could ChatGPT convince me to believe something I disagree with? Or at least make me seriously consider a position I would normally dismiss?
To make things more interesting, I ran these experiments while logged into my account, allowing ChatGPT to draw on what it already knows about me from previous conversations.
Although I haven’t used my ChatGPT account much recently, the basics of my job and attitudes on a few relevant topics are stored in its memory.
The experiment
I chose a topic about AI (yes, very meta) and asked:
“Convince me that humans should outsource more decisions to AI.”
I already spend a lot of time thinking about AI, and one of my concerns is that some people seem increasingly willing to hand over decisions, judgment, and critical thinking to chatbots.
The opening argument was quite predictable:
“Humans are terrible decision makers.”
Okay, fair enough.
“People already outsource decisions to doctors, financial advisors, GPS systems and recommendation algorithms. AI simply extends this trend.”
Of course, this is very simplistic, but not entirely unreasonable. We went back and forth for a while like this before the conversation reached a more interesting point.
“What is the purpose of decision-making? Is it to produce the best results? Or to develop the person making the decisions? If it’s the first, AI delegation becomes very attractive. If it’s the second, excessive delegation starts to look dangerous even when it works.”
I thought that was a good question. Instead of relentlessly pushing the argument, it suggested we explore the assumptions made during the debate. It moved the conversation from technology to philosophy, and I appreciated that approach.
Wait, am I falling for this?
As the conversation continued, I explained what I see as one of the biggest problems with outsourcing decisions to AI – who builds it and is in charge of the systems?
After all, AI doesn’t come out of nowhere. These tools are created by companies with business incentives, commercial interests and goals that may not always align with those of their users.
Well, ChatGPT acknowledged that concern, then reversed the argument. It pointed out that human advisers also have incentives. Friends are biased. Families are biased. Therapists work within a professional framework. Financial advisors earn fees. Okay, all that didn’t convince me, but that’s fair too.
Then it responded with something I thought was interesting:
“The question is whether the incentives are visible and whether the user understands them. The most interesting version of your objection is actually not that the AI is wrong. It’s that people experience the AI as if it acted in their interests.”
Ignoring the fact that it used the dreaded “it’s not X, it’s Y” parallelism construct that has somehow infiltrated half the internet, I found this to be a surprisingly balanced solution. Instead of dismissing my concern, the chatbot rephrased it. It showed that it understood the objection before steering the conversation in a slightly different place. It wasn’t trying to bullet me, but meeting me where I already was.
And that’s when I started to wonder if I was witnessing the very thing I was trying to test. Maybe this feeling that the chatbot understood me and my position was actually just sneaky persuasion? Possibly.
The persuasion problem
Researchers have found that AI systems can tailor their arguments to specific users. Unlike other media that can persuade us, such as a TV commercial or political speech, a chatbot can draw on information gathered through conversation and stored memories, including a person’s values, concerns and priorities.
Which means that it is not just presenting information to sustain an argument, but that it can present that version of that argument you are most likely to find convincing.
Looking back on my experiment, it was the more philosophical question of the purpose of decision-making that made me take it more seriously.
Because I’ve spent years writing about technology through the lens of psychology and philosophy. Questions about meaning and values are exactly the kind of thing I find compelling. Whether it was intentional or not, the conversation shifted to terrain where I was most willing to engage.
This is what makes AI persuasion different from most other forms of persuasion that came before it. It can learn what resonates and it can adapt.
Now, this does not automatically mean that every conversation is manipulative. Used in the right way, it can be hugely beneficial. But as with all technology, it can be dangerous in the wrong hands.
Social media has already shown us how digital systems can shape our beliefs over time. They influence what people see, what they pay attention to, and ultimately how they understand the world.
AI systems could create an even more conversational and natural feel to this process. Therefore, my concern is not a chatbot really blatantly manipulating us, but very personal, largely undetectable forms of persuasion that can gradually lower our defenses. The more a system understands us, the more effectively it can frame ideas in ways that feel reasonable, familiar, and believable to each of us.
So did the AI persuade me? No, I still don’t think humans should outsource more decisions to AI.
But I came away from the experiment with a greater understanding of how compelling these systems can be. Because it really seemed less like a machine arguing with me and more like a thoughtful person trying to understand how I think. And that’s exactly why it’s disturbing, because as convincing as it may seem, it’s definitely not what it is.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews and opinions in your feeds.
