In 2023, 758 consultants at Boston Consulting Group took part in an experiment run with researchers from Harvard, MIT, Wharton and elsewhere. They were an elite group, about 7% of the firm’s individual-contributor consulting staff. Some worked without AI. The rest got GPT-4, at the time the most capable model available.
About half spent the session launching a product for a fictional shoe company, 18 tasks running from brainstorming (“Propose at least 10 ideas for a new shoe targeting an underserved market or sport”) to segmenting the market and writing a press release. With GPT-4 they finished 12.2% more of the work, roughly a quarter faster, and graders rated it markedly better.
The other half got a business case of the kind BCG uses to interview job candidates. A company’s CEO wanted to know which brand had the most room to grow. There was a spreadsheet of sales that looked complete, and a file of interviews with insiders that held the details that changed the answer.
Without AI, consultants got it right about 84.5% of the time. With GPT-4, 60% and 70.6% in the two AI groups.
Same firm, same kind of people, tasks that looked about equally demanding. On one, AI lifted them. On the other, it pulled them down by 19 percentage points on average, and the reports they handed in read better than those of colleagues working alone.
An invisible wall
The researchers named the pattern the “jagged technological frontier.” AI’s abilities cover an uneven patch of knowledge work. “Tasks that appear to human knowledge workers to be of similar difficulty may be performed either better or worse by humans using AI.” Inside the frontier, AI complements people. Outside it, the paper says, “AI output is inaccurate, is less useful, and can degrade human performance.”
Ethan Mollick, one of the authors, asked readers to picture a fortress wall, with towers jutting out into the countryside in some places and folding back toward the castle in others. Everything inside the wall the AI can do. “The problem is that the wall is invisible.” His example from 2023: GPT-4 was good at writing a sonnet and consistently missed when asked for a poem of exactly 50 words.
Better writing, wrong answer
The most unsettling result was about presentation. Graders who didn’t know the correct answer scored every report for clarity, coherence and persuasiveness. Consultants with GPT-4 scored higher, whether or not they had picked the right brand. The paper’s summary: AI “improves presentation and argumentation even when the underlying analysis is flawed.”
Mollick connected this to an earlier experiment by Fabrizio Dell’Acqua, the paper’s lead author, in which recruiters given a very good AI tool grew careless and made worse calls than those given a weaker one. Dell’Acqua called it “falling asleep at the wheel.” When the help is usually right and reliably fluent, the moments it is wrong arrive looking exactly like the moments it is right.
Learning someone’s edges
Managers already know a version of this from people. Imagine a new analyst who joins the team. Her first deck is the clearest anyone has seen. Her manager then gives her something smaller, a reconciliation of two revenue reports, and checks it line by line before it goes anywhere. It is wrong in a way the deck never hinted at. Now the manager knows something the CV didn’t say. Next quarter she will hand over the decks freely and keep checking the numbers for a while.
With a colleague, you learn the shape of their edges once, and it changes slowly. With AI, two things differ. The edges don’t follow human patterns, so a strength in one area predicts little about the task next to it. And the shape can change with each new model.
Drawing the map
So treat the frontier as something to probe, not assume. A first pass can be small, say ten real tasks from last month and an afternoon, though the numbers matter less than the habits. Run each task through the AI you use and have someone who knows the right answers grade the results. Vary the inputs: a messier file, a missing figure, the same question asked another way. Run the important tasks more than once, since the same request can come back different. What you get is a first look at the weak spots, not a finished map. Test again when the model changes.
For tasks near the edge, Mollick described two working styles from the study. Centaurs split the job cleanly, doing the parts they were strongest at themselves and handing the AI tasks inside the frontier. Cyborgs passed pieces back and forth throughout. In both, a person stays in charge of the question the BCG case turned on: which details actually change the answer.
The consultants who got that case wrong had a complete-looking spreadsheet, a fluent assistant and a polished report. They also had the interviews, with the details that changed the answer. The material to check the assistant’s conclusion was in their hands, and it went unused.