What the controlled studies say about AI at work: novices gain most, and the frontier is jagged

In short: the best controlled studies agree on three things. Generative AI raises output most for less experienced people, it improves speed and quality on tasks inside its capabilities, and it makes people worse on tasks just outside them, which they cannot spot by feel. Plan training, measurement and review around those three findings.^1,2,3

Many claims about AI and productivity rest on surveys, often commissioned by vendors. A smaller body of research uses control groups and realistic tasks, and it is the evidence a manager should plan around. Three studies stand out.

Together they say something more useful than “AI makes people faster”: it makes some people much better at some tasks, it makes people worse at others, and people cannot tell which is which by feel.

Customer support: 15 percent on average, more for novices

Brynjolfsson, Li and Raymond (published in the Quarterly Journal of Economics in 2025) studied more than 5,000 support agents at a software company that rolled out a generative AI assistant in phases. Issues resolved per hour rose about 15 percent on average, with the largest gains for novice and lower-skilled agents (34 percent in the working-paper analysis)⁴ and minimal effect on the most experienced. Customer sentiment improved and attrition fell. The authors’ explanation: the assistant spread the practices of the best agents to everyone else.¹

Professional writing: faster and better, with a caveat

Noy and Zhang (published in Science) ran an online experiment that gave 453 college-educated professionals realistic writing tasks. Those with ChatGPT finished about 40 percent faster and produced work rated about 18 percent higher, and the gap between weaker and stronger writers narrowed. The tasks did not require factual precision, and the authors cautioned that real work needs time for checking.²

The jagged frontier

Dell’Acqua and colleagues, researchers from Harvard Business School, Wharton, MIT, Warwick and Boston Consulting Group, ran a pre-registered field experiment with 758 consultants at Boston Consulting Group, on realistic tasks the researchers designed. On tasks inside the model’s capabilities, consultants using AI completed 12.2 percent more tasks, finished 25.1 percent faster and produced work of more than 40 percent higher quality. On a task chosen to sit outside the frontier, AI users were 19 percentage points less likely to reach the correct answer.³ The tool is excellent at some things that look hard and poor at some that look easy.

What the surveys add

Pew Research found in February 2025 that 52 percent of US workers were worried about AI’s future impact on the workplace, 36 percent hopeful and 33 percent overwhelmed.⁵ The World Economic Forum’s Future of Jobs Report 2025 projected 170 million roles created and 92 million displaced by 2030, a net gain of 78 million, with nearly 40 percent of job skills changing.⁶ A Microsoft survey of 1,800 workers, reported in the 2026 Work Trend Index, found that where managers openly model AI use, employees report 17 points more value from AI and 30 points more trust in agentic AI. The survey shows an association, not a cause.⁷

What it means for careers and for managers

• Train new hires first and hardest. Novices gain most, so onboarding is where training pays off soonest.
• Give experts different tools. They gain less from suggestions and more from tools that remove drudgery and protect their judgment.
• Measure quality, not only speed. Speed is where people overestimate the benefit.
• Teach the frontier, not only prompting. The most valuable skill is knowing when to use the tool and when to check it.

For individuals the career implication is the same as the management one. The durable skill is judgment about AI: which tasks it does well in your field, which it does badly, and how to check the difference. Microsoft’s 2025 Work Trend Index named the emerging role the “agent boss,” someone who delegates to and manages agents.⁸ The job is recognizable to any good manager: set the goal and the limits, review the exceptions, and widen autonomy only on evidence.

For the measurement side, our briefing on why productivity numbers mislead sets out what to count instead of self-reports, and the agent boss briefing turns the role into a checklist. Workforce adoption, including who to train first, is taught in Module 10 of the AIMP program.