How many golf balls can you fit into an airplane? How many gas stations are there in Manhattan?
In June 2013, Laszlo Bock, Google’s senior vice president of people operations, recited both questions to Adam Bryant of the New York Times, then gave his verdict. “A complete waste of time. They don’t predict anything. They serve primarily to make the interviewer feel smart.”
He had numbers behind the insult. Years earlier, Bock said, Google had set out to learn whether anyone at the company was particularly good at hiring. It looked at tens of thousands of interviews: who had conducted each one, the score they gave the candidate, and how that person ultimately performed in the job.
“We found zero relationship,” he told Bryant. “It’s a complete random mess.” The one exception was a single interviewer who only saw candidates for a very specialized area, in which, Bock said, he happened to be the world’s leading expert.
What worked better, by Bock’s account, was much plainer: “structured behavioral interviews, where you have a consistent rubric for how you assess people, rather than having each interviewer just make stuff up.” Ask a candidate for an example of a time they solved an analytically difficult problem, then drill into it. You see how they acted in a real situation, he said, and you learn what they consider difficult.
Two years later Bock wrote the same argument into his book, Work Rules!, and added a confession. In an excerpt Wired published in April 2015, he grouped consulting-style case questions with the brainteasers, “most annoyingly, ‘How many golf balls would fit inside a 747?’” Then: “Full disclosure: I’m the Senior Vice President of People Operations at Google, and some of these interview questions have been and I’m sure continue to be used at the company. Sorry about that.”
The replacement looked dull on the page. Bock printed sample questions such as “Tell me about a time your behavior had a positive impact on your team,” each with set follow-ups. One early reader of the draft told him they were “so generic it’s a little disappointing.” Bock’s reply was that the questions are bland and the answers are not: asked the same thing, superb candidates bring much better examples. Then came the rubric. In Google’s version for general cognitive ability, every performance level was clearly defined, and the interviewer had to write down exactly how the candidate had shown it, so later reviewers could make their own assessment.
Google still describes the method in its re:Work hiring guides, with rubrics that spell out what a poor, borderline, solid and outstanding answer would cover.
As for the golf balls, Bock wrote that Google did everything it could to discourage those questions. When senior leaders, himself included, reviewed applicants each week, “we ignore the answers to these questions.”