On January 11, 2018, Mark Zuckerberg posted a long note on his own Facebook page. “I’m changing the goal I give our product teams,” he wrote, “from focusing on helping you find relevant content to helping you have more meaningful social interactions.”
Then he named the cost before anyone could ask. “By making these changes, I expect the time people spend on Facebook and some measures of engagement will go down.”
He had already started down that road. Three weeks later, the quarterly results showed that changes made in the last months of 2017, before the new goal was announced, including showing fewer viral videos, had cut time spent on Facebook by roughly 50 million hours every day. In the US and Canada, the number of people using Facebook daily fell by 700,000 from the previous quarter, and Facebook’s finance chief said the quality changes had contributed. None of that was yet the effect of the new goal. It showed how much he was prepared to let fall.
Pick one number for the value people get, and you’ve also picked what you’ll give up. Zuckerberg had said, in public, which one he was ready to give up. The next question was how a feed with billions of posts would know what “meaningful” meant.
Thirty points for a comment
It knew through a points table. According to internal documents later reviewed by The Wall Street Journal, a like was worth one point. A reaction, or a reshare without any text, was worth five. A significant comment, message, reshare or RSVP was worth thirty. Extra multipliers depended on whether the interaction was between friends, members of a group or strangers.
A ranking algorithm sorted each person’s News Feed. One part of it, called “downstream MSI,” pushed a post higher if the system calculated that people were likely to comment on it or share it as it traveled down a chain of reshares.
In the fall of 2018, BuzzFeed’s chief executive, Jonah Peretti, emailed a top Facebook official. A BuzzFeed post titled “21 Things That Almost All White People are Guilty of Saying” had drawn 13,000 shares and 16,000 comments, many of them people arguing about race. “MSI ranking isn’t actually rewarding content that drives meaningful social interactions,” he wrote.
Facebook’s own researchers found the same pattern. Publishers and political parties had shifted toward outrage and sensationalism, because that drew comments and reactions. One party’s social media team in Poland estimated it had moved from posts that were half positive, half negative to 80 percent negative, “explicitly as a function of the change to the algorithm.” An internal memo put it flatly: “Misinformation, toxicity, and violent content are inordinately prevalent among reshares.”
Nothing in the point table, as the Journal described it, asked what a comment said. The system ranked by the score. People outside the company learned which posts scored.
“Like any optimization, there’s going to be some ways that it gets exploited or taken advantage of,” Lars Backstrom, the vice president of engineering who oversaw feed ranking, told the Journal.
What the star is for
Sean Ellis, Dropbox’s first marketer and a co-author of Hacking Growth (2017), advocates what he calls a North Star Metric. In a 2018 interview he said it is “really trying to quantify that value over time.” His example was Airbnb. Optimize for app installs and the chart can look great while nobody books a room. Nights booked, he said, is what creates value for the guest and the host. For Facebook, he added, “an easier proxy for value is just ‘daily active users.’” That same number had dipped in the US and Canada at the end of 2017.
A north star works like a compass heading on a long walk. It doesn’t tell you where every rock is. It tells everyone in the group which way to lean when the path forks, and that settles a thousand small arguments.
The catch is that a heading is a choice. Facebook’s old goal, in Zuckerberg’s words, was helping people find relevant content, and he said public content led to more overall time spent. Swapping it for interactions meant saying out loud that time spent would go down. Choosing a north star does this whether anyone says so or not: it tells teams which other numbers they’re allowed to let slide.
And it becomes the number people aim at, with everything Goodhart’s Law predicts. The Journal’s documents add one more layer. Inside Facebook, the change had a less publicized motive: comments, likes and reshares had been falling through 2017, and the new formula was meant to reverse that decline. The meaningful-interaction number was built out of engagement.
The number at the top of the whiteboard
Imagine you run product at a language-learning app, and you have to pick the one number that goes at the top of the whiteboard.
Minutes in the app is easy; it rises every time you add a game. Lessons completed sounds better, until someone points out that a two-minute lesson counts the same as a twenty-minute one. You settle on learners who pass the next level’s test, and under it you write what you expect to lose: minutes will probably fall.
Then you add a second number in smaller letters, one that would rise if people started gaming the first: retakes of the same test within a day. If the two climb together, that is something to look into, not a verdict. People may have learned what scores. They may also just be practicing.
Facebook made a version of that move years later. In August 2021, the company said it was “gradually expanding some tests to put less emphasis on signals such as how likely someone is to comment or share political content.” The signals being turned down were the ones its 2018 formula had paid the most points for.