15 points lower, and the commit looked identical.
A pre-registered trial on who does the thinking, not whether AI is "good."
- The track
- Leads a team
- The rep
- About thirty seconds
- Published
- September 2, 2026
The Exhibit
A pre-registered RCT from Anthropic (N=52) split learners by how they used AI on the same task. The group that let the AI drive scored 59% on a comprehension quiz afterward. The group that stayed in the driver's seat scored 74%, a 15-point gap (Cohen's d = 0.74). Identical work product. Different person left holding the understanding. The study.
The headline is not "AI lowers comprehension." The same study found AI used as a tutor, prompting and questioning, matched unassisted learning. The failure mode is specific: passive delegation, letting the model do the thinking. That distinction is what you can actually manage for.
The study
Comprehension Debt
Comprehension debt: the gap between what gets shipped and what the team can still reason about.
Two engineers can ship the identical diff and be in completely different places: one drove and understands it, one delegated and could not rebuild it. The artifact cannot tell you which. Only a question can. This is why review of the code alone keeps missing it: the PR is the output, and the output is exactly what is identical. The signal lives in the conversation, not the diff. At team scale, that is the difference between a group that can defend what it ships and one whose diffs nobody can fully account for. You see it surface as your MTTR drifting up: harder bugs take longer to fix when no one on the team built the model the fix depends on.
The idea
The Rep
One for your next review:
A PR adds input validation with a slick regular expression to check email-like strings. It works on every example in the tests. On a malformed input in staging, one request pins a CPU core for thirty seconds.
A slick regex that passes every test can still take down a box, and the engineer who senses that asks what it does on input that almost matches. Catastrophic backtracking is the name for what they are worried about. "The regex is fine, the tests pass" is the trap, because passing tests only ever prove the happy path: a pattern that is correct on friendly input and lethal on a near-miss is exactly what no one catches by reading the diff, which is why the question has to be asked out loud.
About thirty seconds
