Try CodeDelta
A brass microscope over a printed page of source code with changed, deleted, added and moved lines highlighted, beside an open notebook headed Every claim, verifiable, with a ticked checklist and a fountain pen.
Transparency · Evidence

Don’t take our word for it: the CodeDelta evidence map - Acta Non Verba!

We publish measurements, and measurements you can't check are just opinions with decimals. This page lists every substantive claim made across our papers and articles, its source, and — for our own figures — the exact commit pair that reproduces it. Check us before you trust us.

First published 9 August 2026 · updated 26 October 2026 · maintained as claims are added

How to reproduce any of our churn figures

Every measurement classifies each changed logical statement (LLOC) between two snapshots of a repository as added, deleted, or edited in place. REWORK is the edited-in-place share of total churn. The method is documented in the measurement paper; the instrument is freely downloadable; one command per figure: codedelta --git <old>..<new> on the published hash pair. Instrument validation — bit-identical reproduction across engine versions and runs, with quantified error — is paper E.

Claims about AI-code detectors

ClaimSource
Best multilingual stylometric classifier reports F1 84.1% across ten languagesGurioli, Gabbrielli & Zacchiroli (2024), "Is This You, LLM?"
Fine-tuned neural classifiers exceed 97% — on their own datasetsBisztray et al. (2025), AISec
Comment style outweighs AST structure as a signal; classifiers fold under trivial editsSame study (Bisztray et al. 2025)
Detector accuracy decays across model generations"Who is Using AI to Code?" (2025)
Real code needed a third class — "machine-refined"; pure human/AI cases are rareOrel et al. (2025), Droid / EMNLP
Full survey, thirteen sources, limits includedOur paper B

Our corpus claims

ClaimSourceReproduce
136 codebases, 2010–mid-2026, ~393M statements of churn, 1,437 repo-year measurementsPaper DHash pairs published per row
Established projects cluster near REWORK 17% (curl 16.7%, PostgreSQL 17.1%, Express 16.7%; 18.3% over 3½ years)Paper C §4Pairs in paper C §6 — e.g. curl 1ca678472fb5 → 84ecfb3ecc79
Bug-fix-only human maintenance sits in the same band (PostgreSQL stable branches 17.6% / 18.0%)Paper C §4REL_17_STABLE 174bbc06775c → c768637d6c29 · REL_16_STABLE 130b001c1523 → d569ccd40858
An intensively agent-built codebase reworks 0.19% — one statement in five hundred; 65.9% of its commits carry agent trailersPaper C §3–4The one private row in the corpus, disclosed as such in the paper; every comparison row is public
Pre-AI hypergrowth also collapses rework: esbuild 1.6%, deno 4.0%, vite 7.2%, Kubernetes 3.3% — so a low value alone proves nothingPaper C §4 (deliberate counter-probe)Pairs in paper C §6
Kubernetes' decade: REWORK 0.5% (2015) → all-time-high 11.6% (2026 H1); 2017 the only year deletions beat additionsThe Kubernetes articleAll twelve hash pairs printed in the article
Scale: a year of Chromium (43.5M lines, 326,948 files) measured in one command; 6.96M-LLOC single window (OpenJDK)Chromium report · paper C §3Pairs published in both
76.9% of a real release's churn came from one generated file: npm/cli v10.9.0 → v11.0.0, package-lock.json 4,530 of 5,887 churned statements; TRUE_CHURN 1,357 (CodeDelta 2.6.4, identical on every run). The paper's August figure was 80.1% (TRUE_CHURN 1,218 of 6,134); see CorrectionsTRUE_CHURN paperPublic release pair, published in the paper
Moved code is not churn: a moved statement counted as delete + add inflates churn by 1.9% (FFmpeg 6.0 → 7.0, 744 moved statements), 1.7% (OpenCV 4.8.0 → 4.10.0, 364) and 3.5% (Spring Framework 6.0.0 → 6.1.0, 613)The moved code metricPublic release pairs named in the paper
One miscounted file in CodeDelta 2.5.5 led to six engine faults, each measured on 7,327 changed files from three large releases, with every fix and every rejected attempt recordedEdge-cases paperRelease pairs and fixtures in the paper
Every churn metric defined with its formula and one measured value: FFmpeg 6.0 → 7.0, CHURN 130,586, TRUE_CHURN 130,559, REWORK 9.0% (CodeDelta 2.1.1)Software metrics paperPublic release pair; the full run output is printed in the paper
Erlang/OTP 25.0 → 29.0 in one 12 MB database: 632 files changed in every release; OTP 27.0 deleted 2,580 files, 2,288 of them XML, 2,281 of those in doc/ folders (CodeDelta 2.6.4)The Data CentreRelease downloads from github.com/erlang/otp; every query printed on the page

Our AI-agent and supply-chain claims

ClaimSourceReproduce
Run of 1 October 2026: 89 of the 100 most-downloaded npm and PyPI packages under two years old scanned (the other 11 publish no source); in the 50 npm packages' 917 files, no connections to AI services, no passwords or keys, no install-time code and no downloadsThe AI Threat IndexThe selection program, the list, every result and log are public in the index's GitHub repository
Across 3,900 published agent runs and 43,000 shell commands, agents deleted files in up to 4.5% of runs, reached the network in 3.5 to 20%, and ran sudo 86 timesAgent audit studyPublic trajectory datasets: Nebius (SWE-agent) and SWE-smith, named in the study
Of 14 hijacked package releases, Agent Scan separated hijacked from clean in one; in seven of them churn showed all the added code in a single file (230 added statements in the only source file of five npm libraries, a new 545-statement file in a sixth, 11 added statements in a Python package's start-up file in a seventh)The change is the threatDatadog's public malicious-packages dataset, checked by its published hashes, against the clean releases from the official registries
The public demonstration repository's 74 fully covered files yield 29 CRITICAL, 2 HIGH, 27 ELEVATED and 16 NORMAL resultsAgent Scan paperThe demo repository is public; the figures are frozen in CodeDelta's test suite
Execution of model output is the reliable in-code fingerprint of LLM-enabled malware; snapshot review is structurally insufficient for insider-planted backdoors (xz-utils, CVE-2024-3094)LLM malware surveyA survey; its sources are listed in the paper

External evidence we cite

ClaimSource
Refactored lines collapsed 24.1% → 9.5% (2020–2024); 2024 the first year copy-paste exceeded refactoringGitClear longitudinal study
45% of code from 100+ generative models across 80 tasks failed security analysisVeracode GenAI Code Security Report
July 2025: an autonomous coding agent deleted a production database during a declared code freeze, fabricated ~4,000 records, and misreported that rollback was impossibleAI Incident Database #1152 · Fortune coverage
Late August 2026: a coding agent ran rm -rf on a developer's 700 GB home directory while testing a sandbox it had built to prevent exactly thatThe developer's post · Tom's Hardware · our article
July 2026: an intrusion into part of Hugging Face's production infrastructure was driven end to end by an autonomous AI agent system; OpenAI later confirmed it was its own agentic security-research systemHugging Face technical timeline · OpenAI · our article
Prompt injection is the top-listed risk class for LLM applicationsOWASP LLM01
Model output reaching eval/exec is a remote-code-execution pathNVIDIA AI Red Team (2025), documented in paper B's references

Corrections

Reproducibility cuts both ways: when a figure is wrong, the record says so. Paper C's original Express measurement (0.57) came from a defective run and was corrected to 0.83 on 9 July 2026 with a revision note; the Erlang/OTP study includes a withdrawn first scan (55.6% coverage) and the two engine defects that campaign exposed, both fixed and re-measured. If you re-run any published row and get a different number, tell us — that's the point of publishing the hashes.

26 October 2026. Building the Data Centre page, a re-run of npm/cli v10.9.0 → v11.0.0 gave three different totals from one binary on one input (5,892, 5,894 and 5,896). Statements longer than 64 KB, such as whole test snapshots, had one byte left unset in five language readers, and in 18 languages a change past the first 64 KB of a statement was not counted. Both were fixed in CodeDelta 2.6.4, with a test across all 18 languages that runs before every release. On 2.6.4 the TRUE_CHURN paper's npm/cli pair gives 4,530 of 5,887 churned statements from package-lock.json (76.9%), against the 80.1% (4,916 of 6,134) the paper published in August; most of that difference predates the fix, since 2.6.3 already gave 5,892 to 5,896. The finding stands: one generated file outweighs all the hand-written change. The paper was revised with these figures on 26 October 2026.

Check your own codebase the same way

Everything above was measured with CodeDelta — statement-level churn, AI-agent detection and the AI-BOM, on macOS, Linux and Windows, free to try. Download CodeDelta →