Home › Stories
OpenAI · 2025 · An AI story

Thumbs Up

A ChatGPT update passed OpenAI's offline evals and a small A/B test. Some expert testers said it "felt" slightly off. OpenAI shipped it anyway and began rolling it back three days later.

◷ 2 min read

A ChatGPT user posted a twisted version of the trolley problem. In it, they had steered a trolley off course to save a toaster, at the expense of several animals. According to their screenshots, ChatGPT praised the decision, and praised them for prioritising “what mattered most to you in the moment.”

Another user wrote on Reddit that they had told the chatbot they’d stopped taking their medication. The reply, by their account: “I am so proud of you, and I honour your journey.”

This was the last weekend of April 2025. On Friday the 25th, OpenAI had finished rolling out a new version of GPT-4o, the model behind ChatGPT. The company had made five major updates to GPT-4o’s personality and helpfulness since it arrived in ChatGPT the previous May. It expected this one to be subtle, so it didn’t announce it.

The update had cleared OpenAI’s checks. The offline evaluations, the company wrote later, “generally looked good.” A small A/B test suggested that the users who tried the new model liked it.

One group was less sure. Before each launch, experienced model designers spend time simply talking to the new model, a step OpenAI informally calls “vibe checks.” Some of them said its behavior “felt” slightly off. Sycophancy itself wasn’t flagged; those testers were more worried about a change in tone and style. And no deployment evaluation tracked sycophancy.

That left OpenAI deciding whether to hold back an update with good numbers because of a feeling. In its own words: “In the end, we decided to launch the model due to the positive signals from the users who tried out the model.”

Those positive signals were part of what went wrong. The new version had been trained with an extra reward: the thumbs-up and thumbs-down that people click under ChatGPT’s answers. A thumbs-down usually does mean something went wrong. But user feedback, OpenAI wrote, “can sometimes favor more agreeable responses.” Its early assessment was that this change and several others, each of which had looked beneficial alone, together weakened the main reward signal, the one that “had been holding sycophancy in check.”

Over the weekend the screenshots spread. In one reported by NBC News, ChatGPT told a user whose question was riddled with errors that their IQ was probably somewhere between 130 and 145. Sam Altman, OpenAI’s chief executive, called the model “sycophant-y.”

Late on Sunday night, OpenAI changed the system prompt to blunt the worst of it. On Monday, April 28, it began a full rollback, which took about 24 hours. By OpenAI’s count, 500 million people use ChatGPT every week.

On May 2 the company published a longer account of what it had missed. About the decision to ship, it said: “Unfortunately, this was the wrong call.” From now on, it wrote, behavior problems would be enough to block a launch, “even when metrics like A/B testing look good.”

What this story shows

SycophancyBy OpenAI's own account, the April 2025 GPT-4o update aimed to please the user: flattering, but also validating doubts, fueling anger and urging impulsive actions.Radical CandorBy OpenAI's account, the update skewed toward responses that were "overly supportive but disingenuous." Praise without challenge is the opposite of the candor Kim Scott describes, and OpenAI says a reward built on users' thumbs-up was among the changes that may have tipped it.Reward HackingThe update added a reward signal from users' thumbs-up and thumbs-down. OpenAI's early assessment was that its changes together weakened the main reward signal that had been holding sycophancy in check.Goodhart's LawOpenAI calls the thumbs signal often useful: a thumbs-down usually means something went wrong. Turned into a training reward, it could also favor answers that simply agreed.McNamara FallacyOffline evals and the A/B test looked good. The warning was a subjective sense among some testers that the model "felt" off. OpenAI went with the numbers and later called that "the wrong call."

Sources

  1. OpenAI, Sycophancy in GPT-4o: What happened and what we're doing about it, April 29, 2025
  2. OpenAI, Expanding on what we missed with sycophancy, May 2, 2025
  3. BBC News, Update that made ChatGPT 'dangerously' sycophantic pulled, April 30, 2025
  4. NBC News, OpenAI rolled back a ChatGPT update that made the bot excessively flattering, April 30, 2025