At Google, every interview left a number behind: the score the interviewer gave the candidate. Some years before 2013, by the account of Laszlo Bock, who ran people operations there, the company set those numbers beside something it learned only later: how each hire went on to perform.
Was anyone at Google particularly good at hiring? There were tens of thousands of interviews to check. “We found zero relationship,” Bock told the New York Times in 2013. “It’s a complete random mess.” The lone exception was one interviewer who only saw candidates in a narrow field where, Bock said, he happened to be the world’s leading expert.
All those interviews, and by Bock’s account the scores said nothing about who would do well. The fix he described next sounded too modest to matter.
Ask every candidate the same questions and score the answers on the same scale.
One ruler for everyone
Bock’s version was “structured behavioral interviews, where you have a consistent rubric for how you assess people, rather than having each interviewer just make stuff up.”
The psychologists Frank Schmidt and John Hunter drew the line in a 1998 review of 85 years of hiring research. In an unstructured interview, the same interviewer often asks different applicants different questions. Answers usually aren’t scored one by one, and each applicant gets a single overall rating “based on summary impressions and judgments.” A structured interview is “exactly the opposite on all counts,” with questions that usually come from a careful analysis of the job.
Each change closes a door. When everyone gets the same questions, a gap between two scores is more likely to come from the candidates than from what each happened to be asked. When the scoring guide is written before anyone walks in, a good first impression has less room to quietly redefine what a good answer is. When each answer is scored on its own, one strong moment is less free to color the rest, the pattern behind the halo effect. A baking contest works the same way: one brief, one score sheet, crumb and filling marked separately, instead of a judge who tastes and says “I liked that one.”
Point four two
Schmidt and Hunter ranked hiring methods by validity: the correlation between a method’s scores and later job performance, where 0 means no linear relationship and 1 a perfect positive one. Structured interviews came in at .51, the unstructured kind at .38.
In 2022, Paul Sackett and three colleagues argued that a statistical correction common in such reviews had been applied too generously, and revised the estimates down almost across the board. Structured interviews fell to .42 and unstructured ones to .19. In their words, structured interviews “emerged as the top-ranked selection procedure.” The work sample page sets out more of that table.
A correlation of .42 is no crystal ball. It is an average, and Sackett’s team noted that results for structured interviews varied widely between studies, so one company’s version could land well below it. Done in person, they added, structured interviews generally aren’t practical for high-volume hiring.
Google’s own guide lists the other costs: rubrics must be written, tested and enforced, and questions refreshed so candidates don’t compare notes. It also reports that ready-made questions and rubrics save its interviewers 40 minutes per interview on average.
Three colleagues, one debrief
Say you are hiring a support team lead, and three colleagues will each spend 45 minutes with every finalist. Left to themselves, one asks about the last job, one about a bad week, one mostly about a hobby they share. The debrief can turn into three people defending three impressions, with no way to tell whether the second candidate would have fumbled the question the first one answered so well. Nobody asked them both.
The structured version takes an afternoon before the first interview. You pick the few things the job needs, such as telling a customer no without losing them, and write one question for each, with follow-ups: “Tell me about a time you had to refuse a customer something they were sure they were owed. What did you say? What happened next?” Under each, you note what a poor, borderline, solid and outstanding answer would include, the four levels Google’s guide uses. Each interviewer scores alone, noting what the candidate said, before seeing anyone else’s numbers.
The questions will feel bland. Bock writes that one early reader of his draft called Google’s sample questions “so generic it’s a little disappointing.” The answers are not bland, and now they sit side by side.
Grading the machine
Checking an AI model’s answers is an interview you may run ten thousand times. When the grader is another model, Anthropic’s documentation states the rules in nearly Google’s words. Its guide to building evaluations tells developers to “have detailed, clear rubrics,” with an example: “The answer should always mention ‘Acme Inc.’ in the first sentence. If it does not, the answer is automatically graded as ‘incorrect.’”
A 2026 Anthropic engineering post goes further. “Vague rubrics produce inconsistent judgments,” it says, and a good task is one where two domain experts would independently reach the same pass or fail verdict. It suggests a separate rubric for each dimension, each graded by its own model judge, and an instruction to answer “Unknown” when the judge lacks the information.
What differs is where the risk sits. A human panel needs a rubric to stop each person improvising. A model grader runs the same rubric on every answer, so a flaw in the rubric repeats on every answer too. The post’s answer is to calibrate model graders closely against human experts. The video-editing company Descript, it reports, moved from grading by hand to model graders with criteria set by its product team, plus periodic human calibration.
Write “mentions Acme Inc. in the first sentence,” and that is the line the judge will hold every answer to, whether or not it was the thing that mattered.