For a while, Matt Mullenweg hired the way most companies do. A candidate for Automattic, the company behind WordPress, might meet five employees, and then they would all go out to lunch together. By his account, his team believed they were being as rigorous as they could be.
Some of those hires still didn’t work out. Mullenweg concluded that his team had been grading the wrong performance: the way a candidate talked, even how they behaved in a restaurant. Unless the job is charming people, he wrote in 2014, charm says little about how someone will do it.
So he made a rule that sounds expensive. Every final candidate would work for the company first, for three to eight weeks, on real tasks, next to the people who would become their colleagues. Support candidates answered real customers. Engineers wrote real code. Everyone was paid $25 an hour, the would-be engineer and the would-be finance chief alike.
It sounds like common sense, and for decades the research agreed. Then, in 2022, the research revised its numbers.
The top of the table
In 1998 the psychologists Frank Schmidt and John Hunter pulled together 85 years of studies on how well hiring methods predict job performance. They ranked 19 methods by a correlation called validity. Zero means the method tells you nothing about who will do the job well. One means it predicts perfectly.
Work sample tests came first, at .54. Schmidt and Hunter’s example was an applicant asked to repair a series of broken electric motors, and they noted that such tests were often used to hire welders, machinists and carpenters. Tests of general mental ability and structured interviews tied at .51. The ordinary, free-flowing interview scored .38. Years of job experience, the line most résumés lead with, scored .18.
In 2022 Paul Sackett and three colleagues went back over the evidence behind numbers like these. They argued that a statistical correction common in these reviews had been applied far too generously, inflating many of the estimates. Partly for that reason, and partly because of newer data, their figures came out lower almost everywhere. Work samples fell to .33. Structured interviews, at .42, became the top method. The unstructured interview dropped to .19, and years of experience to .07.
Work samples lost first place but stayed in the leading group. The methods at the top, Sackett’s team noted, are built around a specific job, and on average they beat general measures of ability or personality.
The revision has critics. Three other researchers, Oh, Le and Roth, argued that the problem the correction fixes can matter more in some studies than Sackett’s team allowed. Sackett’s team replied that the conditions where it matters are unlikely to be typical, and conceded that considerable uncertainty remains. Each number is also an average over very different jobs and tests. Yours could land well above it, or well below.
To find out whether someone can do the work, watch them do a small piece of it.
A sign or a sample
Researchers call this the difference between signs and samples. An intelligence test or a personality quiz looks for signs that someone will do well. A work sample takes a small sample of the behavior you are hiring for. A chat over lunch mostly samples how someone talks, the part Mullenweg came to distrust.
There are limits. Work samples suit only people who already know the job, as Schmidt and Hunter pointed out, and they cost far more than an ability test. Automattic’s weeks-long version also sits between a short test and what the 1998 paper listed separately as a job tryout (hiring people on probation and watching them for months), which scored .44.
Two hours with an old client’s books
Say you run a three-person bookkeeping practice and need a fourth. Two candidates interview well. One spent ten years at a large firm and talks fluently about reconciliations. The other is quieter, with four years at a small one.
Instead of a second interview, you take one month of an old client’s books with the names removed: a shoebox of receipts and a bank statement with one duplicate payment in it. You pay each candidate for two hours, hand them the same files and the same instructions, and write down before you look what a good result is: the month reconciled, the duplicate found, and a note on what to ask the client.
The fluent candidate finishes fast and misses the duplicate. The quieter one finds it and lists two questions. One afternoon can’t tell you who will be better over five years. It does show you who caught the mistake your clients pay you to catch, which another hour of talk may not.
Auditioning a model
AI models get picked on something like interviews too: scores on exam-style tests. In 2025 OpenAI wrote that tests like these “often fall short” of the tasks people handle in everyday work. Its response, GDPval, works like a work sample for models: 1,320 tasks across 44 occupations, each built from a real piece of work, such as a legal brief or a nursing care plan, by professionals with 14 years of experience on average. Graders from the same occupations compare a model’s deliverable with the expert’s own, without knowing which is which.
It has the same blind spot as a two-hour test. OpenAI notes that GDPval is one-shot, so it can’t show a model revising a brief after a client’s feedback, or talking to the client first to work out what is needed.
What differs is the price of a sample. A candidate can give you an afternoon. A model can do the same task a thousand times, and Anthropic’s current guide to building evaluations suggests tests that “mirror your real-world task distribution,” with volume ahead of careful hand-grading. Before you hand a model a recurring job, give it twenty real cases from last month whose right answers you already know.
Put the duplicate payment in the pile.