We publish measurements, and measurements you can't check are just opinions with decimals. This page lists every substantive claim made across our papers and articles, its source, and — for our own figures — the exact commit pair that reproduces it. Check us before you trust us.
Every measurement classifies each changed logical statement (LLOC) between two snapshots of a repository as added, deleted, or edited in place. REWORK is the edited-in-place share of total churn. The method is documented in the measurement paper; the instrument is freely downloadable; one command per figure: codedelta --git <old>..<new> on the published hash pair. Instrument validation — bit-identical reproduction across engine versions and runs, with quantified error — is paper E.
| Claim | Source |
|---|---|
| Best multilingual stylometric classifier reports F1 84.1% across ten languages | Gurioli, Gabbrielli & Zacchiroli (2024), "Is This You, LLM?" |
| Fine-tuned neural classifiers exceed 97% — on their own datasets | Bisztray et al. (2025), AISec |
| Comment style outweighs AST structure as a signal; classifiers fold under trivial edits | Same study (Bisztray et al. 2025) |
| Detector accuracy decays across model generations | "Who is Using AI to Code?" (2025) |
| Real code needed a third class — "machine-refined"; pure human/AI cases are rare | Orel et al. (2025), Droid / EMNLP |
| Full survey, thirteen sources, limits included | Our paper B |
| Claim | Source | Reproduce |
|---|---|---|
| 136 codebases, 2010–mid-2026, ~393M statements of churn, 1,437 repo-year measurements | Paper D | Hash pairs published per row |
| Established projects cluster near REWORK 17% (curl 16.7%, PostgreSQL 17.1%, Express 16.7%; 18.3% over 3½ years) | Paper C §4 | Pairs in paper C §6 — e.g. curl 1ca678472fb5 → 84ecfb3ecc79 |
| Bug-fix-only human maintenance sits in the same band (PostgreSQL stable branches 17.6% / 18.0%) | Paper C §4 | REL_17_STABLE 174bbc06775c → c768637d6c29 · REL_16_STABLE 130b001c1523 → d569ccd40858 |
| An intensively agent-built codebase reworks 0.19% — one statement in five hundred; 65.9% of its commits carry agent trailers | Paper C §3–4 | The one private row in the corpus, disclosed as such in the paper; every comparison row is public |
| Pre-AI hypergrowth also collapses rework: esbuild 1.6%, deno 4.0%, vite 7.2%, Kubernetes 3.3% — so a low value alone proves nothing | Paper C §4 (deliberate counter-probe) | Pairs in paper C §6 |
| Kubernetes' decade: REWORK 0.5% (2015) → all-time-high 11.6% (2026 H1); 2017 the only year deletions beat additions | The Kubernetes article | All twelve hash pairs printed in the article |
| Scale: a year of Chromium (43.5M lines, 326,948 files) measured in one command; 6.96M-LLOC single window (OpenJDK) | Chromium report · paper C §3 | Pairs published in both |
| 80.1% of a real release's churn came from one generated file (npm/cli v10.9.0 → v11.0.0; TRUE_CHURN 1,218 of 6,134) | TRUE_CHURN paper | Public release pair, published in the paper |
| Claim | Source |
|---|---|
| Refactored lines collapsed 24.1% → 9.5% (2020–2024); 2024 the first year copy-paste exceeded refactoring | GitClear longitudinal study |
| 45% of code from 100+ generative models across 80 tasks failed security analysis | Veracode GenAI Code Security Report |
| July 2025: an autonomous coding agent deleted a production database during a declared code freeze, fabricated ~4,000 records, and misreported that rollback was impossible | AI Incident Database #1152 · Fortune coverage |
| Prompt injection is the top-listed risk class for LLM applications | OWASP LLM01 |
Model output reaching eval/exec is a remote-code-execution path | NVIDIA AI Red Team (2025), documented in paper B's references |
Reproducibility cuts both ways: when a figure is wrong, the record says so. Paper C's original Express measurement (0.57) came from a defective run and was corrected to 0.83 on 9 July 2026 with a revision note; the Erlang/OTP study includes a withdrawn first scan (55.6% coverage) and the two engine defects that campaign exposed, both fixed and re-measured. If you re-run any published row and get a different number, tell us — that's the point of publishing the hashes.
Everything above was measured with CodeDelta — statement-level churn, AI-agent detection and the AI-BOM, on macOS, Linux and Windows, free to try. Download CodeDelta →