Anton Dziatkovskii · ORCID 0000-0001-7408-3054
Preprint, 10 September 2026. Version of record: Zenodo, DOI 10.5281/zenodo.22684968. License CC BY 4.0.
Full text (PDF, 6 pages, 298 KB)DOICode: compact-canon
Abstract. Long-running LLM coding agents survive finite context windows by compaction: replacing the conversation with a model-written summary. Compaction is lossy, and practitioners are widely advised to steer it by writing “compact instructions” into a project configuration file or a pre-compaction hook. We measure whether those instructions arrive. On two machines of one production fleet we scanned complete agent transcript archives and, for every compaction event, joined the summary to its metadata, recovered what the operator actually passed to the compaction command, and scored whether the summary carried the operator's custom seven-header skeleton.
On machine A (16,107 transcript files; 456 events; CLI 2.1.161–2.1.246) not one of 447 compactions run without an inline instruction produced the skeleton. On machine B, measured for this paper (10,435 files; 469 events; CLI 2.1.149–2.1.260), 3 of 431 did (0.7%, 95% CI 0.2–2.0%) — and in all three the header text was already in the conversation before compaction, so the format was copied from visible content rather than delivered as an instruction. Passing the same instructions inline, the documented path, produced the skeleton in 16 of 38 runs on machine B and 5 of 9 on machine A (pooled 21/47, 44.7%, 95% CI 31.4–58.8%): often, not deterministically. Seeded live probes on the current release reproduce both halves.
The paper also reports, with fixtures, four defects that make the naive version of this measurement report successes that are not there: the substring detector we began with calls 82 of 335 bare compactions successes where the corrected one finds 2. The study is single-fleet, Windows-only, one agent product; detector, fixtures and per-event table are public so the numbers can be contested.
Code and data: github.com/tonydzi/compact-canon · tonydzi.github.io · contact dzyatkovskiy.a2@gmail.com · ORCID 0000-0001-7408-3054.
LLM agents · context compaction · context window management · agent memory · prompt engineering · empirical software engineering · measurement methodology · Claude Code · reproducibility
Dziatkovskii, A. (2026). Instructions That Never Arrive: Measuring Whether Operator Compaction Instructions Reach the Summarizer in a Production LLM Coding Agent. Preprint. Zenodo. https://doi.org/10.5281/zenodo.22684968
@misc{dziatkovskii2026compactcanon,
author = {Dziatkovskii, Anton},
title = {Instructions That Never Arrive: Measuring Whether Operator Compaction Instructions Reach the Summarizer in a Production LLM Coding Agent},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.22684968},
url = {https://doi.org/10.5281/zenodo.22684968},
note = {Preprint, CC BY 4.0}
}
One text, one DOI: the PDF on this page is the Zenodo file byte for byte. Please cite the DOI.