Skip to content
AtomicReps
The Retrieval · Issue 05

15 points lower, and the commit looked identical.

A pre-registered trial on who does the thinking, not whether AI is "good."

The track
Leads a team
The rep
About thirty seconds
Published
September 2, 2026

The Exhibit

A pre-registered RCT from Anthropic (N=52) split learners by how they used AI on the same task. The group that let the AI drive scored 59% on a comprehension quiz afterward. The group that stayed in the driver's seat scored 74%, a 15-point gap (Cohen's d = 0.74). Identical work product. Different person left holding the understanding. The study.

The headline is not "AI lowers comprehension." The same study found AI used as a tutor, prompting and questioning, matched unassisted learning. The failure mode is specific: passive delegation, letting the model do the thinking. That distinction is what you can actually manage for.

The study

Comprehension Debt

Comprehension debt: the gap between what gets shipped and what the team can still reason about.

Two engineers can ship the identical diff and be in completely different places: one drove and understands it, one delegated and could not rebuild it. The artifact cannot tell you which. Only a question can. This is why review of the code alone keeps missing it: the PR is the output, and the output is exactly what is identical. The signal lives in the conversation, not the diff. At team scale, that is the difference between a group that can defend what it ships and one whose diffs nobody can fully account for. You see it surface as your MTTR drifting up: harder bugs take longer to fix when no one on the team built the model the fix depends on.

The idea

The Rep

One for your next review:

A PR adds input validation with a slick regular expression to check email-like strings. It works on every example in the tests. On a malformed input in staging, one request pins a CPU core for thirty seconds.

A slick regex that passes every test can still take down a box, and the engineer who senses that asks what it does on input that almost matches. Catastrophic backtracking is the name for what they are worried about. "The regex is fine, the tests pass" is the trap, because passing tests only ever prove the happy path: a pattern that is correct on friendly input and lethal on a near-miss is exactly what no one catches by reading the diff, which is why the question has to be asked out loud.

About thirty seconds

Start the sequence

You read this one out of order. Start at issue one.

Subscribe and the sequence begins at the beginning, whichever issue brought you here.

Which track

One confirmation email first. Nothing else until you click it.