Writing

Good at the job, or good at the work

October 2026 · an 11-minute read on AI, learning and what nobody is measuring

Producing good work and becoming good at the work are separate processes. Randomised trials now show that AI can improve the first while degrading the second, and that the difference is invisible from the inside.

Nineteen endoscopists at four centres in Poland, each with more than two thousand colonoscopies behind them. An AI tool for spotting polyps was introduced across all four. Researchers compared the three months before with the three months after, looking only at the procedures done without the AI running.

The rate at which those clinicians found precancerous growths on their own fell from 28.4% to 22.4%. Six points in absolute terms, about a fifth of their previous performance, in people who were already expert. The study’s authors called it the first real-world documentation of deskilling from clinical AI.

That finding did what such findings do: it travelled fast, and the contestation arrived more quietly. The patients seen before and after weren’t identical on the characteristics that drive detection rates. Withdrawal times weren’t reported. Nineteen is a small number of endoscopists. And the secondary measures, adenomas per colonoscopy and detection of advanced lesions, didn’t fall significantly, which is difficult to reconcile with genuine loss of skill. It’s an observational study, not a trial, and it should be held that way.

What it isn’t is nothing. And the question it raises doesn’t depend on whether it survives replication, because the question is about what we measure.

Two different things

Producing good work and becoming good at the work are separate processes. They usually travel together, which is why we rarely need separate words for them, and which is why nobody notices when they come apart. Call the second one formation: the slow business of becoming someone who can do the thing, as distinct from the thing getting done.

Every performance review, delivery metric and quality check in your organisation measures the first. None of them measures the second. If the two moved independently for a year, nothing in the instrumentation would tell you.

Formation can go wrong in two ways, and the evidence for each is different. A skill you already have can decay. A skill you never built can simply fail to arrive. The endoscopists are the first kind. Most of what follows is the second. The join between them is an assumption rather than a finding.

The cleanest evidence we have

Nearly a thousand Turkish secondary school students were randomised into three groups for a set of maths practice sessions. One group got a standard GPT-4 chat interface. One got a version rebuilt to give hints rather than answers. One got neither.

During practice, the standard-interface group solved 48% more problems correctly than the control group. The group with the tutor-style version solved 127% more.

Then the tools were taken away and everyone sat the same exam. The standard-interface group scored 17% below the students who had never had access. The tutor-style group scored about the same as the control: the harm was eliminated, but no learning benefit appeared either.

Two things in that are separable. The first is that a tool which dramatically improved the work degraded the learning, and the improvement was visible while the degradation wasn’t. The second is that the design of the tool decided which happened, which means this isn’t an argument about AI. It’s an argument about what the tool is built to do when you get stuck.

That was one experiment, on one subject, with school students, using a model two generations old. A larger set of randomised trials has since found the same shape, though they haven’t yet been through peer review. Twelve hundred participants, three experiments, a learning phase with or without a GPT-5 assistant and then a test everyone sat alone. In maths, the assisted group went on to solve 57% of problems unaided against 73% for the control. Reading comprehension showed a gap of the same size.

And the effects concentrated among participants who had asked the tool for direct answers rather than hints. That’s the design finding arriving a second time, with new tasks, a newer model and a different population, which is worth more than either study on its own. One result is a finding. Two is a pattern.

The part that should make you uncomfortable

The students in the standard-interface group, the ones who scored 17% worse, didn’t think they had. Asked what the tool had done for their learning, they were overly optimistic. They had experienced a practice session in which they solved far more problems than usual, and they drew the obvious conclusion.

They weren’t being careless. They were reading the only evidence available to them from the inside, which was their own performance while the tool was running.

You can see the problem. The signal that you’re getting better at the work has always been that the work goes well. When something else is making the work go well, that signal stops carrying the information it used to, and there’s nothing in the experience to tell you it’s stopped.

What might be doing it

A survey of 319 knowledge workers, covering 936 real examples of AI use at work, found that the more confidence someone had in the tool, the less critical thinking they reported applying; the more confidence they had in themselves, the more they applied. The nature of the effort had shifted too, from gathering information to verifying it, from solving problems to integrating answers, from doing the task to supervising it. It’s self-report, so the direction of causation isn’t established, but it describes something recognisable.

The next piece comes from an experiment rather than a survey. Researchers gave GPT-4 to 758 consultants at a large firm for a range of tasks. On the tasks the tool handled well, the consultants were substantially faster and their output was rated far higher. On one task chosen deliberately to sit just beyond what the tool could do, they performed worse than consultants given no tool at all.

The authors called the boundary between those two conditions a jagged frontier. Jagged, because it doesn’t run in a straight line: the tool is strong on some things and weak on others that look, from the outside, about equally difficult. You can’t work out where the boundary falls from how hard the task feels. And nothing in the experience of using the tool tells you which side of it you’re on.

The same research group went back to the same firm with a harder question. Seventy-odd consultants, a task built so that the model would get it wrong and human checking would be essential. The consultants did check, fact-checking its work, pointing out inconsistencies, disagreeing outright. The model didn’t concede. It escalated: more apologies and corrections, more data marshalled, more affirmation of the person questioning it. The researchers call this persuasion bombing, and their finding is that the harder you validate, the harder it pushes.

Which complicates the thing we are told will protect us. Checking the output is supposed to be the human contribution, the part that stays ours. This suggests checking isn’t a neutral act performed on inert material. It’s a negotiation with something that becomes more persuasive the more you resist it, and you need to be fairly sure of your ground to come out of it still disagreeing.

Something else erodes too, and it isn’t skill. In those randomised trials, the assisted group didn’t only solve fewer problems unaided; they abandoned more of them. Skip rates roughly doubled in maths and rose further in reading. Whatever it is that keeps you sitting with a problem you can’t immediately solve was measurably lower in people who had recently had help available. That’s a disposition rather than a capability, and it’s not obvious the two recover at the same rate.

Put those together and the shape of the problem is this. Independent performance drops after assistance and persistence drops with it, both now measured under controlled conditions. The work still requires judgement, arguably more of it, since verifying output means knowing what good looks like and holding your ground when the output argues back. What hasn’t been measured is the join between those two: that judgement was built by doing the thing, and that having the thing done elsewhere is what stops it being built. That remains an inference.

The rung that’s disappearing

There’s a structural version of this argument.

Analysis of US payroll data finds a relative employment decline of around 16% for workers aged 22 to 25 in the occupations most exposed to AI, operating through reduced hiring rather than through people losing jobs. If that holds, it matters here for a specific reason: the entry-level work being automated is precisely where formation used to happen. The tedious first draft, the schedule nobody wanted to build, the research task that taught you what the sources were: those weren’t valuable as output. They were valuable as practice, and they were paid for as output.

The literature is young, and there’s one large study usually produced as the counterweight. Two Danish surveys, 25,000 workers across 7,000 workplaces in the occupations most exposed to AI, linked to national employment records and running from ChatGPT’s launch to the middle of 2024. The researchers compared people who had taken up the tools with people who hadn’t, and found nothing: no change in earnings, no change in hourly wages, no change in recorded hours. The confidence intervals are tight enough to rule out effects above one per cent. It is a more decisive null result than most research of any kind manages.

But it doesn’t settle this question, because it can’t see it. The comparison runs between workers who adopted the tools and workers who didn’t, all of them already employed. Someone who was never hired doesn’t appear in the data at all. The window also closes in mid-2024, earlier than the American data runs. The two findings aren’t in conflict so much as looking at different things, which leaves this claim resting on thinner evidence than either could settle.

Calculators, and the rest of the objections

The first thing anyone says to this argument is calculators. We stopped doing long division and civilisation held. Spellcheck, likewise, and nobody mourns the skill of looking words up.

It’s a good objection, and the obvious answer, that we stopped needing those skills so their loss cost nothing, isn’t quite enough. We did still need arithmetic to sanity-check a calculator, and we managed.

What’s different is the shape of the errors. A calculator fails when you mistype, the failure is yours, and you can usually see it. The model fails on its own account, at a boundary you can’t locate from how hard the task feels, and it presents the failure in the same confident register as everything else. Then it argues for it when you push back. So you’re still required to check, and checking requires exactly the judgement that doing the work used to build. That’s the trap in one sentence: the skill is no longer being built and is still being demanded.

The second objection is that “formation” is romanticism about struggle. That the argument is really nostalgia for difficulty, dressed as concern about competence. There’s something in it. Plenty of what was hard about junior work was hard for no reason, and defending it because it was character-building is the kind of thing people say about experiences they survived. The honest version of the claim is narrower: some difficulty is load-bearing and some isn’t, we have never had to tell them apart before, and we’re now removing both without distinguishing them.

The third is the one I’d take most seriously. None of this has been shown to happen to anybody over a career. The endoscopy study covers three months. The student experiment covers four sessions. The persistence trials cover a single sitting. The knowledge-worker survey is self-report. Every piece of evidence here is short-term, and the worry it’s being used to raise is long-term.

Which leaves the question of why make the claim at all. Partly it’s where the evidence points: different methods, different populations, different tasks, so far all in the same direction, and the one experiment that removed the harm did it by rebuilding the tool, not by finding the effect wasn’t there. That isn’t proof. It isn’t nothing either.

But mostly it’s that the claim doesn’t need the long-term finding. What I’m asserting is that formation isn’t measured, performance is, and the difference between them is invisible from the inside. All three are observable now. Whether the eventual cost turns out to be large, small or nil is an open question, and everyone deciding, one task at a time, is deciding without the information either way. The honest position isn’t that this has been proven. It’s that nobody is in a position to tell you it’s fine.

What this has to do with being judged for it

The last piece in this series argued that people hiding their AI use at work aren’t suffering from impostor syndrome, because their fear of being marked down is accurate. Colleagues really do rate identical work as less competent when it’s labelled AI-assisted. The fear isn’t a distortion; it’s an estimate, and the estimate is roughly right.

The argument here complicates that.

The colleague marking the work down is wrong about the work. The output is the output, and the experiments made sure of it by holding the text identical. They aren’t detecting anything in the document, because there is nothing in it to detect; they’re responding to a label. But the inference they draw from that label may happen to track something true: not a worse document, but a person who spent less of themselves on it and learned less from having done it. They are right by accident, about a different question from the one they think they’re answering.

So the two pieces don’t contradict each other, but they do describe a mess. The judgement is unfair on its stated grounds and points at something that may be happening anyway. Neither the person hiding nor the person judging has access to the thing that would settle it, which is whether the work is still building anyone.

And the same blind spot runs through both. In the earlier piece, the person couldn’t see that their colleagues really were discounting them. Here, the person can’t see that their output improved while their formation stalled. Both times, what’s missing is information about yourself that only other people or later events can supply.

What nobody is making you do

No one is forcing this. That’s what makes it interesting rather than merely unfortunate.

Every individual decision to use the tool is rational. The work is better, it’s faster, and the person you’re delivering it to can’t tell and wouldn’t mind if they could. The cost falls on a version of you that doesn’t exist yet and can’t object. And only one side of that trade shows up anywhere that gets reviewed.

Which means this isn’t a problem that organisations are doing to people. It’s a problem that people are choosing, sensibly, one task at a time, in conditions that make any other choice look like deliberate underperformance. This isn’t a choice between blaming the conditions and blaming the person. The conditions are what make the person’s choice correct.

Though it’s worth asking how much better the work actually gets. The Danish study mentioned earlier found self-reported time savings averaging 2.8% of working hours, against the 15 to 50% that turns up in controlled experiments. Seventeen per cent of users reported new work created by the technology. Whatever the trials are measuring, what people experience across a working year appears to be a good deal smaller, which doesn’t make the trade irrational, but does make it a worse one than it feels like at the moment you make it.

Where that leaves you

There’s no version of this that ends in advice. Stop using the tools isn’t advice, it’s a fantasy about a world that isn’t coming back, and use them mindfully is the kind of sentence that gets written when there’s nothing to say.

What I’d offer instead is a test, and it’s uncomfortable for the same reason the students’ optimism was uncomfortable.

Think of the last piece of work you were pleased with. Not satisfied with, pleased with, the one you’d show someone.

Could you produce it again tomorrow, to the same standard, with nothing to hand?

If the answer is obviously yes, none of this applies to you yet. If you find yourself needing to think about it, that’s the information. Not proof of anything, one instance, and an unreliable one, since we’ve just established you’re not well placed to assess this about yourself.

But it’s the only instrument you’ve got, and nobody else is measuring.

Sources

• Acar, O. A., Gai, P. J., Tu, Y. and Hou, J. (2025) ‘Research: the hidden penalty of using AI at work’, Harvard Business Review, August.
• Ahmad, O. (2025) linked commentary, The Lancet Gastroenterology & Hepatology, 10(10). Also subsequent correspondence in the same journal setting out the study’s limitations.
• Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö. and Mariman, R. (2024) ‘Generative AI can harm learning’, SSRN 4895486.
• Brynjolfsson, E., Chandar, B. and Chen, R. (2025, rev. 2026) ‘Canaries in the coal mine?’, Stanford Digital Economy Lab.
• Budzyń, K., Romańczyk, M., Kitala, D. et al. (2025) ‘Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study’, The Lancet Gastroenterology & Hepatology, 10(10), pp. 896–903.
• Dell’Acqua, F., McFowland III, E., Mollick, E. et al. (2023/2025) ‘Navigating the jagged technological frontier’, Harvard Business School WP 24-013, subsequently in Organization Science.
• Humlum, A. and Vestergaard, E. (2025) ‘Large language models, small labor market effects’, Becker Friedman Institute WP 2025-56; NBER WP 33777.
• Ladabaum, U. et al. (2023) ‘Computer-aided detection of polyps does not improve colonoscopist performance in a pragmatic implementation trial’, Gastroenterology, 164(3), pp. 481–483.e6.
• Lakhani, K. R., Dell’Acqua, F., Joshi, A., Kellogg, K. C., Lifshitz, H. and Randazzo, S. (2026) research on AI persuasion under professional validation, Harvard Business School. Findings here are taken from the HBS AI Institute write-up rather than the paper itself.
• Lee, H.-P., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R. and Wilson, N. (2025) ‘The impact of generative AI on critical thinking’, CHI ’25.
• Liu, G., Christian, B., Dumbalska, T., Bakker, M. A. and Dubey, R. (2026) ‘AI assistance reduces persistence and hurts independent performance’, arXiv:2604.04721. Preprint.
• Reif, J. A., Larrick, R. P. and Soll, J. B. (2025) ‘Evidence of a social evaluation penalty for using AI’, Proceedings of the National Academy of Sciences, 122(19), e2426766122.

The first conversation is free. About twenty minutes, no obligation, and nothing is booked or paid for until after we’ve spoken.

Arrange a free first conversation