The regex passed every test and pinned a core.
59% vs 74%, same task, different person in the driver's seat.
- The track
- Writes code
- The rep
- About thirty seconds
- Published
- September 2, 2026
The Exhibit
Anthropic ran a pre-registered RCT (N=52) on the same learning task, split by interaction style. Let the AI drive: 59% on the comprehension quiz. Stay driving yourself: 74%. A 15-point gap, Cohen's d = 0.74. The study.
The nuance matters: the same paper shows AI-as-tutor can match learning on your own. Passive delegation is the failure mode, not the tool. Which means the move is not to stop using the model, it is to stay the one asking the questions.
The study
Comprehension Debt
Comprehension debt: the gap between what gets shipped and what the team can still reason about.
You can accept a block of generated code, watch the tests go green, and ship something you could not have written and cannot fully defend. Recognition, not recall. The bill arrives when it breaks in a way the tests never covered and the only person who can debug it is the one who understood it, which today was the model, not you.
The idea
The Rep
Thirty seconds:
A PR validates emails with a clever regex. Green on every test. In staging, one malformed input pins a CPU core for thirty seconds. What happened?
The answer
Catastrophic backtracking, a regex denial-of-service. Patterns with nested or overlapping quantifiers (think `(a+)+$` shapes, or naive `(.*)*`) can force the engine into exponential backtracking on input that almost matches. The tests only fed it inputs that matched cleanly, so the exponential path never ran. Fixes: avoid nested quantifiers and ambiguous alternations, anchor and bound the pattern, prefer a real parser or a linear-time engine (RE2) for untrusted input, and never trust a regex on adversarial strings just because it is green on friendly ones. The model will happily hand you the dangerous pattern; the tests will happily pass it.
About thirty seconds