Paper · Edge cases

Moved statements v changed statements: edge cases - precision metrics - no fuzzy guesstimates

Saying a tool measures moves and churn is a sentence. Showing the cases it got wrong, how they were found, and what is still wrong is the measurement.

24 October 2026 · figures measured with CodeDelta 2.5.5 and 2.6.3 on FFmpeg 6.0 → 7.0, OpenCV 4.8.0 → 4.10.0 and Spring Framework 6.0.0 → 6.1.0

A churn tool sorts every changed statement into one of four categories: changed in place, deleted, added, or moved. The categories are easy to name. Where a tool goes wrong is in deciding which category a statement belongs in when the change is awkward: an edit with a new line inserted above it, a function that moved and was edited in the same commit, a short statement that travelled with its block, a file path whose slashes changed inside a string. This paper starts from one small file that CodeDelta 2.5.5 miscounted, follows six faults into the engine, measures each one on 7,327 changed files from three large releases, and records every fix with the evidence it had to pass, including the attempts that were rejected.

How the counting works

CodeDelta compares two versions of a file twice: once by physical lines (SLOC) and once by logical statements (LLOC). For the statement pass each language reader turns the file into a list of statements, with whitespace removed and comments stripped, so if (total > 100.0) return 0.0; becomes the token if(total>100.0)return0.0;. A longest-common-subsequence trace lines up the statements that are identical in both versions. Everything else falls into gaps: a run of statements deleted from the old version facing a run of statements added in the new one.

Inside a gap, statements are paired by position: the first deletion faces the first addition, and so on. A facing pair counts as one changed statement (CHG) if at least 30% of the identifier words in the two statements are shared; otherwise it is one deletion (DEL) and one addition (ADD). A statement deleted in one place whose identical text is added somewhere else is a move (MOV), reported beside churn, never inside it. Churn* is CHG + DEL + ADD.

* CRN_LLOC = CHG_LLOC + DEL_LLOC + ADD_LLOC. REWORK = CHG ÷ CRN, the share of churn that edited existing statements. The move rule is set out in Moved code is not churn; statements versus lines in Why LLOC is what really counts.

Every fault below sits in one of those steps: what the reader puts in a statement’s text, how the gap pairs its statements, and what counts as a move.

The example

The code churn page works through pricing.c, seventeen lines with five separate edits: discount() reformatted across lines, the free-shipping threshold in shipping() lowered from 100.0 to 75.0, a new tax() function, shipping() moved below print_receipt(), and printf("Thank you\n"); removed. Measured one edit at a time, each came out right. With all five in one file the right answer is one changed, one deleted, two added, one moved: churn 4.

tax() has nothing to do with the move or the edit, so where it sits should not change the answer. It did:

Where tax() sits	CHG	DEL	ADD	MOV	Churn, 2.5.5	Churn, 2.6.3
After discount()	0	2	3	1	5	4
First in the file	1	1	2	1	4	4
Before shipping()	0	3	4	0	7	4
Last in the file	0	3	4	0	7	4

The CHG to MOV columns are 2.5.5’s. Three placements, three different wrong answers. In each, the one-word edit 100.0 → 75.0 scored two (a deletion and an addition) instead of one. In the last two the move of shipping() was not recorded at all.

Fault 1: an edit compared only with the line facing it

Positional pairing compares a deletion with the addition at the same offset in its gap. If that one fails the 30% test, both are written off and the walk moves on. The edited statement one place further along is never compared. No move is needed to trigger it:

old                                   new
keep_one_statement(total);            keep_one_statement(total);
if (total > 100.0) charge(0.0);       log_value(total);
keep_two_statement(total);            if (total > 75.0) charge(0.0);
                                      keep_two_statement(total);

Right answer: one changed, one added. CodeDelta 2.5.5: none changed, one deleted, two added. The if faced log_value(total);, shared no words with it, and was counted as deleted.

The same blindness also produced wrong pairings when the offsets happened to line up badly. In Spring’s FreeMarkerMacroTests.java, checks written as assertThat(getMacroOutput("FORM2"))… were rewritten as testMacroOutput("FORM2", …). 2.5.5 paired them by offset: form 2’s old check with form 1’s new one, form 4 with form 3, form 14 with form 5, and so on down the file, each change counted against the wrong statement.

Fault 2: an edit whose original sits in a different gap

When a function moves and is edited, the trace may keep a neighbour in place, so the edited statement and its original land in different gaps. 2.5.5 only paired across that distance when a recognised move sat directly beside the slot. In pricing.c the only candidate was return 4.99; (fault 3). On FFmpeg’s libavcodec/av1dec.c the same thing split real edits that had moved:

old  ret = ff_cbs_read_packet(s->cbc, &s->current_obu, pkt);
new  ret = ff_cbs_read_packet(s->cbc, &s->current_obu, s->pkt);

old  av_log(avctx, AV_LOG_DEBUG, "Total obu for this frame:%d.\n", ...);
new  av_log(avctx, AV_LOG_DEBUG, "Total OBUs on this packet: %d.\n", ...);

Each pair is one statement edited, over 220 statements apart. 2.5.5 counted each as a deletion and an addition.

Fault 3: a short line that moved

Since 16 August 2026 a statement only counted as moved if its text was at least twelve characters with whitespace removed and appeared exactly once among the file’s deletions and once among its additions. The limit is there for a reason: without it, one trial run recorded 5,313 moves that never happened, as unrelated break; and return; statements in the same file paired with each other. But it judges a statement by length alone. return 4.99; is eleven characters, so the unchanged half of the moved shipping() could not count as moved, and with no move beside it, fault 2 followed.

Fault 4: lines that cannot be measured counted as changed

The 30% word test needs at least two identifier words on each side. A statement such as return ret; becomes the single word returnret once whitespace is gone, and for single-word statements the test returned “similar” by default. So whatever faced a one-word statement in a gap was counted as an edit of it. From FFmpeg’s af_silenceremove.c and Spring’s QosSettings.java:

return ret;          paired with   av_frame_free(&out);
return false;        paired with   return this.deliveryMode * 31 + this.priority;
int val_;            paired with   static const ConvParam_t testConvolution_3x3S1D1 ...

Fault 5: characters dropped inside strings

The C-family reader, used for C, C++, Java, C#, PHP and Rust, dropped some characters whenever they appeared inside a string literal: characters with a special meaning outside strings had no path that kept them inside one. The statement open_path("/etc/passwd", 0); was stored as open_path("etcpasswd",0);, so an edit that only changed slashes in a path or URL was invisible:

Language	Change "/etc/passwd" → "etcpasswd", 2.5.5
C, C++, Java, C#, PHP, Rust	0 statements changed (1 line changed)
JavaScript, TypeScript, Go, Python	1 statement changed

To find every such character rather than the one that happened to be noticed, two C files were generated for each of the 95 printable ASCII characters, identical except for that character inside a string. 2.5.5 lost only those with a special meaning outside strings. The engine had a test for slashes inside strings (const char *u = "http://x";), and it passed throughout, because it checked only that the file still counted three statements, not that the slashes were still in the text.

Fault 6: a slash directly before a quote

Outside a string, a / that did not open a comment was stored together with the character after it. If that character was a quote, the reader never entered string state, so a ; inside the string ended the statement early and everything after it merged:

int y = n/"ab;cd"[0];     a statement count of 1 for 3 statements
int y = n/';';            a statement count of 1 for 3 statements
int y = n / "ab;cd"[0];   correct: with a space after the slash

The same shortcut stored the space after a slash when there was one, so a reformat around a slash looked like an edit. OpenCV’s bundled libtiff, tif_getimage.c, reformatted b = (k*(255-pp[2]))/255; to b = (k * (255 - pp[2])) / 255;: identical code, which 2.5.5 counted as a changed statement.

How often, on real code

Every changed file in the three release pairs, 7,327 files in all, was run through the engine on its own, and the statements it left as deleted and added were searched for near-duplicates: statements that differ in only a few characters. Each such pair is very likely one statement edited and counted twice.

Release pair	Edits counted as edits, 2.5.5	Edits split into DEL + ADD	Share of edits	Churn overstated by
FFmpeg 6.0 → 7.0	16,679	1,565	8.6%	3.1%
OpenCV 4.8.0 → 4.10.0	13,829	966	6.5%	2.5%
Spring Framework 6.0.0 → 6.1.0	11,746	866	6.9%	2.5%

Of 24 hits read at random, 20 were plainly one statement edited, for example OpenCV’s libtiff moving from uint16 samplesperpixel; to uint16_t samplesperpixel;; two were different declarations that looked alike; two could not be judged from the text. Moves missed by the twelve-character rule were bounded the same way: at most 331, 121 and 383 statements, most of them common fragments such as return 0; and for(int i=0; that probably did not move.

The fixes, and the evidence each had to pass

Every change was made on a separate branch and had to pass three checks before the next was tried: the full test suite (152 tests); a before-and-after run of the whole engine on the three release pairs, listing every file whose counts changed; and a random sample of the statements the change newly paired, or unpaired, read one by one. The sample matters most. A change that pairs more statements always makes the totals look better, right or wrong.

Step	Rule	Sample read	Outcome
1	Look further along the gap for a partner, using the ordinary 30% word test	40 new pairs: 30 right, 6 wrong, 4 unclear	Rejected
2	Same, but only for near-identical pairs that are clearly each other’s match	40 new pairs: 36 right, 3 wrong (from the older positional rule after the shift), 1 unclear	Kept
3	Same test across the whole file, for edits whose original sits in another gap	40 new pairs: 35 right, 1 wrong, 4 doubtful	Kept, relabelled (below)
4	A short statement that travels with code that moved counts as moved	40 new moves, each read in its file with three lines either side: 39 real, 1 doubtful	Kept
5	Every character inside a string kept as text; a plain / no longer swallows the next character	95-character sweep: 0 lost; 14 new test cases	Kept
6	Single-word statements judged by their characters instead of “similar” by default; equally good partners settled consistently	30 pairs removed: 25 were wrong, 3 real edits lost, 2 debatable. 20 pairs added: 17 right, 3 wrong	Kept

Step 1 failed on pairs such as CV_EMDTest test; with an unrelated call to calcEMD, and two OpenCV assignments paired crosswise, each with the other’s edited version:

attrs.coordinate_transformation_mode = ngraph::...   paired with   attrs.mode = ov::...
attrs.mode = ngraph::...                             paired with   attrs.coordinate_transformation_mode = ov::...

Step 2’s stricter conditions exist for that case. It also un-paired the Spring form checks of fault 1, which had been wrong all along.

Step 3 first labelled its pairs “changed and moved”. On FFmpeg that count rose from 46 to 819. Reading them showed many had not moved at all: static VLC j_dc_vlc[2][8]; → static const VLCElem *j_dc_vlc[2][8]; in intrax8.c sits at statement 1 in both versions. A different gap is not evidence of a move, so the label was removed and the pairs are counted as plain edits.

Step 4 judges a short statement by where it went, not by its length alone. The moves it found are the kind that travel with code: Spring’s ResourceHttpRequestHandlerTests.java moved a group of tests into a nested class, so testResolvePathWithTraversal(location, "url:" + secretPath); went from line 403 to line 700 with its neighbours; FFmpeg’s av1dec.c moved a function whose first statement is AV1DecContext *s = avctx->priv_data;, a statement that appears fourteen times in the new file and could never pass a uniqueness rule. The doubtful one, MuxStream *ms = ms_from_ost(ost); in ffmpeg_mux.c, was paired between two different loops on the strength of a neighbour that was itself an uncertain edit.

Step 6 lost three real edits, each a pair that shares too few characters once whitespace is gone: pushpic eax → PUSHPIC eax in OpenCV’s assembly, int id; → enum AVCodecID id; in FFmpeg’s dashenc.c, and one ff_inlink_consume_frame call. It removed 25 wrong ones like those under fault 4. Its tie rule settles the commonest leftover: several declarations renamed the same way in one file, as libtiff did across its files when uint16 became uint16_t (uint16 plane; → uint16_t plane; in tif_read.c), where an old statement had two equally good partners.

The result

All four placements of tax() give churn 4 with one statement moved. The no-move example gives one changed, one added. Every printable character inside a C string now counts. On the three release pairs:

Release pair	Churn, 2.5.5	Churn, 2.6.3	Moved, 2.5.5	Moved, 2.6.3	REWORK, 2.5.5	REWORK, 2.6.3
FFmpeg 6.0 → 7.0	130,586	128,621	834	973	9.0%	10.5%
OpenCV 4.8.0 → 4.10.0	75,044	74,114	402	485	19.3%	20.4%
Spring Framework 6.0.0 → 6.1.0	51,928	50,991	673	816	22.6%	24.3%

Churn falls by 1.2% to 1.8%. REWORK rises by 1.1 to 1.7 points, because edits that had been split now count as edits. Measured the same way on both engines, from the statements each leaves as deleted and added, the near-identical pairs still split fall from 2,551 to 259 on FFmpeg, 1,136 to 69 on OpenCV and 1,209 to 71 on Spring.

What is still not right

A small number of statements are left unpaired on purpose, where pairing would mean guessing; the three real edits lost in step 6 and the one doubtful move in step 4 are known errors in the other direction.