We publish measurements, and measurements you can't check are just opinions with decimals. This page lists every substantive claim made across our papers and articles, its source, and — for our own figures — the exact commit pair that reproduces it. Check us before you trust us.
Every measurement classifies each changed logical statement (LLOC) between two snapshots of a repository as added, deleted, or edited in place. REWORK is the edited-in-place share of total churn. The method is documented in the measurement paper; the instrument is freely downloadable; one command per figure: codedelta --git <old>..<new> on the published hash pair. Instrument validation — bit-identical reproduction across engine versions and runs, with quantified error — is paper E.
| Claim | Source |
|---|---|
| Best multilingual stylometric classifier reports F1 84.1% across ten languages | Gurioli, Gabbrielli & Zacchiroli (2024), "Is This You, LLM?" |
| Fine-tuned neural classifiers exceed 97% — on their own datasets | Bisztray et al. (2025), AISec |
| Comment style outweighs AST structure as a signal; classifiers fold under trivial edits | Same study (Bisztray et al. 2025) |
| Detector accuracy decays across model generations | "Who is Using AI to Code?" (2025) |
| Real code needed a third class — "machine-refined"; pure human/AI cases are rare | Orel et al. (2025), Droid / EMNLP |
| Full survey, thirteen sources, limits included | Our paper B |
| Claim | Source | Reproduce |
|---|---|---|
| 136 codebases, 2010–mid-2026, ~393M statements of churn, 1,437 repo-year measurements | Paper D | Hash pairs published per row |
| Established projects cluster near REWORK 17% (curl 16.7%, PostgreSQL 17.1%, Express 16.7%; 18.3% over 3½ years) | Paper C §4 | Pairs in paper C §6 — e.g. curl 1ca678472fb5 → 84ecfb3ecc79 |
| Bug-fix-only human maintenance sits in the same band (PostgreSQL stable branches 17.6% / 18.0%) | Paper C §4 | REL_17_STABLE 174bbc06775c → c768637d6c29 · REL_16_STABLE 130b001c1523 → d569ccd40858 |
| An intensively agent-built codebase reworks 0.19% — one statement in five hundred; 65.9% of its commits carry agent trailers | Paper C §3–4 | The one private row in the corpus, disclosed as such in the paper; every comparison row is public |
| Pre-AI hypergrowth also collapses rework: esbuild 1.6%, deno 4.0%, vite 7.2%, Kubernetes 3.3% — so a low value alone proves nothing | Paper C §4 (deliberate counter-probe) | Pairs in paper C §6 |
| Kubernetes' decade: REWORK 0.5% (2015) → all-time-high 11.6% (2026 H1); 2017 the only year deletions beat additions | The Kubernetes article | All twelve hash pairs printed in the article |
| Scale: a year of Chromium (43.5M lines, 326,948 files) measured in one command; 6.96M-LLOC single window (OpenJDK) | Chromium report · paper C §3 | Pairs published in both |
76.9% of a real release's churn came from one generated file: npm/cli v10.9.0 → v11.0.0, package-lock.json 4,530 of 5,887 churned statements; TRUE_CHURN 1,357 (CodeDelta 2.6.4, identical on every run). The paper's August figure was 80.1% (TRUE_CHURN 1,218 of 6,134); see Corrections | TRUE_CHURN paper | Public release pair, published in the paper |
| Moved code is not churn: a moved statement counted as delete + add inflates churn by 1.9% (FFmpeg 6.0 → 7.0, 744 moved statements), 1.7% (OpenCV 4.8.0 → 4.10.0, 364) and 3.5% (Spring Framework 6.0.0 → 6.1.0, 613) | The moved code metric | Public release pairs named in the paper |
| One miscounted file in CodeDelta 2.5.5 led to six engine faults, each measured on 7,327 changed files from three large releases, with every fix and every rejected attempt recorded | Edge-cases paper | Release pairs and fixtures in the paper |
| Every churn metric defined with its formula and one measured value: FFmpeg 6.0 → 7.0, CHURN 130,586, TRUE_CHURN 130,559, REWORK 9.0% (CodeDelta 2.1.1) | Software metrics paper | Public release pair; the full run output is printed in the paper |
Erlang/OTP 25.0 → 29.0 in one 12 MB database: 632 files changed in every release; OTP 27.0 deleted 2,580 files, 2,288 of them XML, 2,281 of those in doc/ folders (CodeDelta 2.6.4) | The Data Centre | Release downloads from github.com/erlang/otp; every query printed on the page |
| Claim | Source | Reproduce |
|---|---|---|
| Run of 1 October 2026: 89 of the 100 most-downloaded npm and PyPI packages under two years old scanned (the other 11 publish no source); in the 50 npm packages' 917 files, no connections to AI services, no passwords or keys, no install-time code and no downloads | The AI Threat Index | The selection program, the list, every result and log are public in the index's GitHub repository |
| Across 3,900 published agent runs and 43,000 shell commands, agents deleted files in up to 4.5% of runs, reached the network in 3.5 to 20%, and ran sudo 86 times | Agent audit study | Public trajectory datasets: Nebius (SWE-agent) and SWE-smith, named in the study |
| Of 14 hijacked package releases, Agent Scan separated hijacked from clean in one; in seven of them churn showed all the added code in a single file (230 added statements in the only source file of five npm libraries, a new 545-statement file in a sixth, 11 added statements in a Python package's start-up file in a seventh) | The change is the threat | Datadog's public malicious-packages dataset, checked by its published hashes, against the clean releases from the official registries |
| The public demonstration repository's 74 fully covered files yield 29 CRITICAL, 2 HIGH, 27 ELEVATED and 16 NORMAL results | Agent Scan paper | The demo repository is public; the figures are frozen in CodeDelta's test suite |
| Execution of model output is the reliable in-code fingerprint of LLM-enabled malware; snapshot review is structurally insufficient for insider-planted backdoors (xz-utils, CVE-2024-3094) | LLM malware survey | A survey; its sources are listed in the paper |
| Claim | Source |
|---|---|
| Refactored lines collapsed 24.1% → 9.5% (2020–2024); 2024 the first year copy-paste exceeded refactoring | GitClear longitudinal study |
| 45% of code from 100+ generative models across 80 tasks failed security analysis | Veracode GenAI Code Security Report |
| July 2025: an autonomous coding agent deleted a production database during a declared code freeze, fabricated ~4,000 records, and misreported that rollback was impossible | AI Incident Database #1152 · Fortune coverage |
Late August 2026: a coding agent ran rm -rf on a developer's 700 GB home directory while testing a sandbox it had built to prevent exactly that | The developer's post · Tom's Hardware · our article |
| July 2026: an intrusion into part of Hugging Face's production infrastructure was driven end to end by an autonomous AI agent system; OpenAI later confirmed it was its own agentic security-research system | Hugging Face technical timeline · OpenAI · our article |
| Prompt injection is the top-listed risk class for LLM applications | OWASP LLM01 |
Model output reaching eval/exec is a remote-code-execution path | NVIDIA AI Red Team (2025), documented in paper B's references |
Reproducibility cuts both ways: when a figure is wrong, the record says so. Paper C's original Express measurement (0.57) came from a defective run and was corrected to 0.83 on 9 July 2026 with a revision note; the Erlang/OTP study includes a withdrawn first scan (55.6% coverage) and the two engine defects that campaign exposed, both fixed and re-measured. If you re-run any published row and get a different number, tell us — that's the point of publishing the hashes.
26 October 2026. Building the Data Centre page, a re-run of npm/cli v10.9.0 → v11.0.0 gave three different totals from one binary on one input (5,892, 5,894 and 5,896). Statements longer than 64 KB, such as whole test snapshots, had one byte left unset in five language readers, and in 18 languages a change past the first 64 KB of a statement was not counted. Both were fixed in CodeDelta 2.6.4, with a test across all 18 languages that runs before every release. On 2.6.4 the TRUE_CHURN paper's npm/cli pair gives 4,530 of 5,887 churned statements from package-lock.json (76.9%), against the 80.1% (4,916 of 6,134) the paper published in August; most of that difference predates the fix, since 2.6.3 already gave 5,892 to 5,896. The finding stands: one generated file outweighs all the hand-written change. The paper was revised with these figures on 26 October 2026.
Everything above was measured with CodeDelta — statement-level churn, AI-agent detection and the AI-BOM, on macOS, Linux and Windows, free to try. Download CodeDelta →