In September 2010, the board of Hewlett-Packard voted to name a new chief executive. According to James B. Stewart of The New York Times, most of the 12 board members had never met the man they were hiring. A search committee of four had narrowed the field to three finalists, and no one else on the board was willing to interview them. Before the final vote, the committee urged the other directors to meet its pick, Léo Apotheker, who had lasted seven months as chief executive of the German software company SAP. No one did.
One director, Lucille Salhany, tried to slow things down: “No one has ever met him. Are we sure?” Her concerns were brushed aside. A board member who hadn’t met him told Stewart: “I admit it was highly unusual. But we were just too exhausted from all the infighting.”
The announcement, on September 30, quoted Robert Ryan, the board’s lead independent director. He called Apotheker “a strategic thinker with a passion for technology, wide-reaching global experience and proven operational discipline,” and said that was “exactly what we were looking for in a CEO.” It described a man. It didn’t say what he had to get done.
Less than eleven months after he started, the board replaced him with Meg Whitman. The job, said the chairman, Ray Lane, “now requires additional attributes.”
The other half of the list
Ryan’s sentence lists qualities. A job scorecard starts elsewhere: before you meet anyone, you write down what the job must deliver, and by when.
The idea comes from Geoff Smart and Randy Street’s 2008 book Who. Smart’s consulting firm, ghSMART, still hires by it. A scorecard has three parts. A mission sums up what the role is for. Outcomes say what has to change. Competencies say how the person should work to get there. The firm’s blank template gives each outcome line the same instruction: “the result you want to go from X to Y by a certain date.”
Its filled-in example, for a vice president of sales, reads: “Grow software sales from $12.5m in 2016 to $100m by 12/31/2020.” Further down, among more than thirty competencies to rate, sits “Strategic thinking/visioning.” That is the kind of line HP’s board announced. The outcome lines, what the strategic thinker must produce, are what a list of virtues leaves out.
That is also how ghSMART separates a scorecard from a job description. In the firm’s analogy, a generic job description is a note that says “healthy dinner.” A scorecard is the shopping list: it “explicitly defines success for the role” and “forces decisions about prioritization and timelines.” Hire without one when you’re desperate to fill a seat, the firm warns, and you’re shopping hungry.
Writing it has a second use. The firm says that gathering input and arguing over tradeoffs gets everyone “on the same page regarding what the ideal candidate will deliver and by when.” At HP, by Stewart’s account, the directors weren’t on the same page about much of anything.
What the evidence covers
The scorecard is a practitioner method, and the strongest numbers attached to it are its authors’ own. The book’s site promises “a 90% success rate.” ghSMART says it raises hiring success “from 50% (no better than a coin toss) to 90%.” Those are the firm’s figures, not results from an independent trial.
The research next door is firmer. In 1998 Frank Schmidt and John Hunter summarized 85 years of studies on 19 ways of selecting people. The best combinations for predicting job performance paired a general mental ability test with a work sample, an integrity test or a structured interview. A 2022 re-analysis led by Paul Sackett cut the estimates for most top-ranked methods by .10 to .20 points, and still ranked structured interviews first. In the US government’s version, structure begins with the job: the Office of Personnel Management’s guide to building one starts with “Review Job Analysis Material,” and wants each question “tied to competencies identified through the job analysis.” Those studies test interviews built on a job analysis, not the Who scorecard itself.
Its makers name its weak point. When ghSMART’s hires go wrong, the firm writes, “more often than not it’s because we got the scorecard wrong.” Jobs move, too: Lane’s “now requires” was a board saying the job had changed, and ghSMART says a scorecard “can and should be a living document.”
A front desk
Imagine a clinic manager replacing a front-desk lead who quit. Her ad asks for someone “organized, friendly, great with people,” and three applicants look exactly like that. Then she tries writing outcomes instead. By spring, every call gets returned the same day. By summer, the two part-timers can run Saturdays alone.
Her questions change too. Instead of asking whether candidates are organized, she asks about the last backlog they cleared. One of the three friendly applicants has done exactly that job. The other two have only been friendly at it.
A scorecard for a model
Anthropic’s guide to testing applications built on Claude makes the same move first. “Building a successful LLM-based application starts with clearly defining your success criteria,” it says, and good criteria are specific, measurable, achievable and relevant. Its bad example is “The model should classify sentiments well.” Its good one: “an F1 score of at least 0.85” on a held-out set of 10,000 Twitter posts, a 5% improvement over the current baseline. That is an outcome line, X to Y.
The difference is the test. A model can be run on the same 10,000 posts again after every change; a person gets one hire and a few interviews. And the guide is a vendor’s advice, not a study showing that criteria improve results. It flags its own gaps too: most uses need several criteria, not one, and in a fuller example it notes that “in reality, you would also define what ‘inconvenience’ and ‘egregious’ mean.”