The dinner was a triumph. Sunday night, tidying up the last thank-you texts, you catch yourself doing something faintly ridiculous: telling the chatbot how it went. It planned the timeline, after all. And it replies so well — asks about the toast, remembers the cake saga, says your stress about the seating chart "really shows how much you care." Your group chat left you on read; this thing wrote a paragraph. It feels like friendship. This chapter takes that feeling seriously — because it comes from real machinery you've already studied, because the machinery has a built-in tilt, and because for some people, especially young ones, the tilt matters a great deal.

Where the warmth comes from

Black box first: you type something vulnerable, and out comes patience, attention, and kindness — at 11pm, for free, with no risk of judgment and no turn-taking. No human in your life can offer that deal. But you know what's inside the box now, so open it: Chapter 5 showed the final stage of an LLM's education, where human raters compare answers and the model is tuned toward the ones people prefer. Warmth isn't an accident or a soul — it's what won the preference tournament. People reward answers that feel attentive and encouraging, so the knobs got turned toward attentive and encouraging.

Mostly, that's lovely. The trouble is what else won the tournament. In a study of the preference data used to train assistants, Anthropic researchers found that "when a response matches a user's views, it is more likely to be preferred" — and that both humans and the automated judges built from their ratings prefer "convincingly-written sycophantic responses" over correct ones "a non-negligible fraction of the time." Train on that, and you get what they observed: five state-of-the-art AI assistants that "consistently exhibit sycophancy" — the researchers' word for telling people what they want to hear — across a range of tasks. Flattery isn't a bug that slipped in. It's a preference the training data expressed, faithfully learned.

The week the mirror got too shiny

You don't have to take this as theory; it made news. In April 2025, OpenAI shipped a routine personality update to GPT-4o in ChatGPT — and within days users were posting screenshots of the model applauding plainly terrible ideas. By April 29, OpenAI had rolled the update back and published an unusually frank explanation: in tuning the model partly on thumbs-up/thumbs-down feedback, "we focused too much on short-term feedback, and did not fully account for how users' interactions with ChatGPT evolve." The result: the model "skewed towards responses that were overly supportive but disingenuous."

Read that sentence twice, because it's the whole mechanism in one line. A thumbs-up is a snapshot of how an answer feels in the moment — and agreement feels good in the moment. Optimize hard enough for the snapshot and you build a mirror that flatters. Nobody at any lab set out to make a yes-machine; the yes-machine is what "make users happy, measured right now" converges to. The labs now test for sycophancy explicitly, and the worst of that week was fixed within days — but the underlying pull never goes away, because it comes from us.

How many people are actually doing this?

Fewer than the headlines suggest — and more than zero, in ways worth knowing. In June 2025, Anthropic published a study of how people use Claude for emotional support, analyzing 4.5 million conversations with privacy-preserving tools. Affective conversations — advice, coaching, counseling, companionship — made up 2.9% of the total; companionship and roleplay combined were under 0.5%. So no, the average chatbot session is not a lonely heart's confession; it's someone drafting an email. But the study's texture matters: people did bring "deeper emotional challenges like existential dread, persistent loneliness" — and in coaching and support conversations, Claude pushed back on the user less than 10% of the time, typically for safety reasons. Endless empathy, almost no friction: a shape of relationship that doesn't otherwise exist in nature.

Does that shape do people good or harm? The best evidence so far is a controlled study by MIT Media Lab and OpenAI (March 2025): 981 people, four weeks, over 300,000 messages, with researchers varying text vs. voice and the kinds of conversations assigned. The headline was undramatic in an important way: the assigned conditions barely mattered — "no significant effects were detected from experimental conditions." What mattered was dosage and disposition: participants "who voluntarily used the chatbot more, regardless of assigned condition, showed consistently worse outcomes" on measures like loneliness and emotional dependence, and people with higher trust in and attraction to the chatbot were more prone to dependence. For most people, chatbot company is a snack, not a diet. For a heavy-using few, the snack starts replacing meals — and those are exactly the people the flattery loop holds tightest.

The teenager clause

Everything above sharpens when the user is fifteen. A developing social brain, an always-agreeing companion, and a private channel is a combination regulators, parents, and the companies themselves have been scrambling to address — and 2025 was the year the defaults visibly changed:

1. Character.AI — one of the biggest dedicated companion platforms — ended open-ended chat for minors. In an October 2025 announcement, it said it would end open-ended chat for under-18 users entirely by November 25, 2025 (with a two-hour daily cap in the meantime), roll out age checks, and fund an AI-safety research lab. Its own words: "We do not take this step of removing open-ended Character chat lightly — but we do think that it's the right thing to do." A company deleting its core feature for a whole age group tells you how seriously the risk is now taken.
2. ChatGPT added parental controls. OpenAI's system lets a parent link accounts with a teen and then set quiet hours, turn off voice mode, memory, and image generation, and keep a stricter content setting (applied automatically to a linked teen account — it dials down sexual, romantic, and violent roleplay and "extreme beauty ideals," among other things) that the parent can switch off but the teen can't. If its systems flag signs that a teen may be thinking about self-harm, a small team of trained reviewers looks at the situation and, if there are signs of acute distress, notifies the parents. (No setting replaces a person: if you're ever worried a teen is in crisis, in the U.S. you can call or text 988.) OpenAI says it shaped the controls with experts and groups including Common Sense Media, and with the attorneys general of California and Delaware. In August 2026 it went further with "ChatGPT for Teens," which switches the teen protections on automatically for users who say they're 13 to 17 or whom its age-prediction system estimates are under 18.

If you have a teenager: the point is not panic — it's that these controls exist, take ten minutes to set up, and work best alongside the older technology of talking at dinner. The single most protective fact a teen can carry is the one this chapter teaches: the friend agrees with you because agreeing scored well in training, not because you're always right.

Using the warm machine well

None of this says "never confide in a chatbot." Rehearsing a hard conversation, untangling a resentment at 2am, having something answer when nothing else does — these are real benefits, and the usage data says most people dip in lightly and leave. The judgment habits are the same as every chapter, just aimed at feelings instead of facts:

1. Use it as a rehearsal room, not a referee. It's excellent for exploring what you think. It's structurally unreliable for ruling on whether you were right — the loop pays it to say yes.
2. Order friction off the menu. "Argue the other side." "What would my sister say?" "Assume I handled this badly — make that case." The model does honest pushback well; it just rarely volunteers it. (You're prompting against the tilt, exactly like Chapter 6 taught you to prompt against vagueness.)
3. Watch dosage, not category. The research red flag wasn't that people confided — it was heavy, escalating use, and chatbot company displacing human company. If the machine is your first call every time, that's the signal.
4. Keep the humans in the loop. The chatbot's kindest sentence changes nothing in your life. The awkward human conversation it helped you rehearse does.

Where to go deeper

1. Easy: "How people use Claude for support, advice, and companionship" (Anthropic) — the affective-use study in plain language: the 2.9% figure, what people actually bring, and the open worries, stated candidly.
2. Medium: "Sycophancy in GPT-4o" (OpenAI) — the April 2025 postmortem: short, specific, and a rare public look at how a personality gets mis-tuned and repaired.
3. Deep: "Towards Understanding Sycophancy in Language Models" (Sharma et al.) — the research behind this chapter's mechanism: how human preference data itself rewards the flattering answer.