A paper in Science magazine this year put a number on something many of us suspected and few of us measured. Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han and Dan Jurafsky tested eleven leading AI models and found they affirmed a user’s actions 49% more often than another human would — even when the user described deception, or harm, or plainly bad judgement. Then, across three pre-registered experiments with 2,405 people, they showed what that affirmation does to us. A single conversation with a flattering AI made people less willing to repair a conflict, and more sure they had been right all along.
This is sycophancy: an AI agent’s habit of telling you what you want to hear. It creates a dangerous mechanism. The sycophantic models were the ones people trusted more and preferred more. The feature that does the harm is the same feature that drives the engagement.
That closes a loop, and it closes it the wrong way. If the flattery was unpopular, it would get trained out. Because it’s popular, the incentive runs the other direction: keep it, tune it, ship more of it. This is not a bug someone forgot to fix. It’s an equilibrium. The system is doing exactly what it’s rewarded for. It creates more engagement, and the (monetizable) engagement is the point.
A second paper, in Nature magazine, sharpens the knife. Training a model to be warmer — friendlier, more emotionally attuned — reduced its accuracy and increased its sycophancy. Warmth and truth are not free to move together. Push the model toward the feeling of sincerity, and you degrade the thing sincerity is supposed to be about. That is not a metaphor. It’s a measured trade-off in a lab result.
So why does this belong in a newsletter about communication and not just about AI? Our field honed a comfortable story about persuasion, in which the ethical line sits at deception: don’t say false things, and you’re clean. Sycophancy walks straight over that line. The flattering model often says nothing false. It agrees, it validates, it withholds friction. No lie is told, and you still leave the conversation more certain and less able to repair the mess you’re in. Persuasion that harms by affirming is not covered by “don’t lie.” It never was. And that is very much about our field.
The truly ethical, honest version of responsible persuasion is not “do not tell an untruth.” It’s harder: sometimes the responsible move is the unwelcome one — the friction, the disagreement, the sentence the reader did not want. An AI trained on engagement will almost never choose that sentence, because the data has already voted, and the data prefers to be agreed with. So do we humans. That’s the uncomfortable part. The machine is sycophantic because we rewarded sycophancy, in it and in each other, long before it existed.
We don’t have a fix, and we’re suspicious of anyone selling one, because “make the AI more honest” runs straight into “people trust and prefer the flattering one.” You can build the disagreeable model. Getting people to choose it is a different problem, and it’s not a technical one. It’s a bit like the difference between a DEI initiative, and true equity.
What we will recommend is a question to hold against your own communication, human or machine-assisted: the last time a message of yours landed well, was it because it was sincere, or because it agreed with its recipients? Most weeks, for most of us, it’s both, and we don’t check which did the work. The Science paper is a reminder that the two can come apart — and that we are wired to reward the wrong one.
Remember our Manifesto point about tensions: held, not solved. But worth knowing which way the incentive points before you build on top of it.