Compaction Probe: CI tests for provenance loss when agents summarise their own history

No company I found sells a CI product that forces compaction and checks whether planted text comes back as a user instruction. The research on the open-source side, though, is already about as specific as this product idea. The Compaction Laundering paper covers the attack, a test across nine harnesses, published code, and a code-level fix. So the open probe suite is not new as an idea. What a new entrant could add is turning it into a maintained product. That means a harness-agnostic CI action, attack strings for files, web pages, tool output and MCP that get updated as harnesses change, regression tracking across versions, and patches upstreamed into the five vulnerable harnesses (pi, opencode, Cline, Aider, OpenHands). The risk is that this gets copied easily. Promptfoo or PyRIT could add it as a plugin, and the paper's authors at Pixee already hold the credibility. The open tool works better as a way to win customers than as something people pay for. The fintech model-risk audit has more room. Nobody I found maps compaction testing to model-risk documentation such as SR 11-7-style validation records. Fintech buyers pay for evidence and repeatability, not novelty, so the second idea is probably the stronger business, and the open suite is what earns it credibility. A big caveat is that the attack only works on harnesses that flatten the conversation into one string. Codex, Gemini CLI and the other role-typed harnesses already scored 0 percent, so the market shrinks as harnesses switch to typed roles. To stay relevant past that point, the product would have to cover softer failures too: lost safety constraints, the summarizer being told what to drop, and the model writing instructions into its own summaries.

• Compaction Laundering paper (Arshan Dabirsiaghi, Pixee)

  This is the closest thing to the probe-suite idea. The paper plants forged [User]: lines in content the agent reads, then measures whether they survive compaction as real user instructions. Across five harnesses that flatten the conversation into a single string (pi, opencode, Cline, Aider, OpenHands), the forged approval survived in 87 to 100 percent of trials. Four role-typed harnesses (Codex, Gemini CLI, Qwen Code, Continue) showed 0 percent. It proposes a code-level provenance check and escaping role labels before flattening. The abstract says code and data are published, so this is effectively an open research version of the suite and the reference patch. Posted very recently, not peer reviewed, and I did not find a product or CI package built from it.

• Governance Decay paper

  Academic study of a compaction-eviction attack. Attacker content either pads the history to force compaction or tells the summarizer what to drop, so safety constraints quietly disappear. It looks at constraints being lost rather than attacker text being promoted to user instructions, but uses the same attack surface and the same force-compaction method.

• OpenAI Alignment report on self-generated injections in compaction summaries

  Shows a big lab is watching compaction summaries for instruction-like content, in this case content the model wrote itself. Labs could build this testing in-house for their own harnesses, which makes them less likely to buy it.

• opencode compaction issues (#27037, #54697) and V2 checkpoint framing

  Harness maintainers are already filing and fixing compaction provenance problems one at a time: compaction requests that look like ordinary user turns, and the full tool set being exposed to the summarizer. One analysis says opencode V2 now wraps the summary as historical context. These are ad-hoc fixes, not a shared test suite.

• Slipstream: trajectory-grounded compaction validation

  Research on checking whether compaction summaries are faithful to the agent's actual history. It is aimed at summary quality rather than security, but it is nearby work that could grow into provenance checks.

• MemPoison benchmark

  Benchmark for poisoning an agent's persistent memory. It sits next to compaction laundering but covers memory stores, not compaction inside a session.

• Unit 42 indirect prompt injection into long-term memory via summarization

  Proof of concept in which a web page manipulates the agent's session summarization so attacker instructions get stored in memory and carry over to later sessions. Shows that security vendors are aware of summarization as an attack path.

• Promptfoo / PyRIT / garak (general red-teaming tools)

  These are the general CI and red-teaming tools. Promptfoo's database already lists cases where an attacker's text loses its origin by passing through agent summarization. None of the three appears to ship a test that forces compaction and checks provenance, but they are the most likely to add one as a plugin.

Checked 2026-10-22