A candidate at Automattic, the company behind WordPress, might once have met five employees, and then all of them would take the candidate out to lunch.
That was how Matt Mullenweg hired in the company’s early years, after he founded it in 2005. His team screened résumés. They sometimes asked brainteasers. They paid special attention to which start-ups a candidate had worked for. By his account, they put a lot of energy into it and believed they were being as rigorous as they could.
Some of those hires didn’t work out anyway. When that happens, Mullenweg wrote, you have to do some soul-searching. Managers at other companies seemed to shrug, as if it were normal for a third of new hires to wash out.
Looking back, he decided the problem was what the process measured. His team had been swayed by a candidate’s manner of speaking, even by how they behaved in a restaurant. Some people charm everyone in an interview. Unless the job is charming people, he wrote, that skill says little about how they will do.
So the lunches gave way to tryouts. After a résumé screen and a first interview, usually by online chat, every final candidate worked for Automattic on contract for three to eight weeks. Most put in 10 to 20 hours a week, on nights and weekends, so they could keep their current jobs. A would-be support person answered real customers. An engineer wrote real code. For business roles, where nobody could hand a stranger a live deal, the team built the closest task it could: a presentation, an analysis, a set of numbers to run.
Everyone was paid the same rate, $25 an hour, whether they were trying out as an engineer or as the chief financial officer. At first the company had set each tryout’s rate by what the candidate might earn if hired, and found itself negotiating pay with people who had no offer yet.
The team watched for things a lunch can’t show: whether candidates got going on their own, how they wrote, since most of the company worked remotely, and what they did when they made a mistake. Overseeing a tryout came before a person’s regular work, and one employee could handle only two or three at once.
By Mullenweg’s account in 2014, about 40% of the people who tried out were hired. Candidates were measured against a standard, not against each other, so if ten strong people auditioned, all ten could get offers.
Scott Berkun, a former Microsoft executive who led a team at Automattic, compared it to ordering a wedding cake. You ask the baker for a sample. “You don’t ask him to talk about his cake.”
Today’s version is smaller. Automattic’s hiring page now describes, for some roles, a short test, or a longer trial on real work that most candidates finish within a week or so. The pay has not changed: $25 an hour, whatever the job.