# Impact Is Not Invalidation: Ask About the Claim, Not the Diff

arXiv: https://arxiv.org/abs/2609.25130 · HTML: https://mlsanity.com/paper/2609.25130/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 20 Sep 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 6 of 6)

- [Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge](https://mlsanity.com/paper/2606.26511/) (2026)
- [Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Review](https://mlsanity.com/paper/2106.13891/) (2021)
- [ChangeGuard: Validating Code Changes via Pairwise Learning-Guided Execution](https://mlsanity.com/paper/2410.16092/) (2024)
- [Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrieval](https://mlsanity.com/paper/2605.11325/) (2026)
- [SemaDiff: Identifying Semantic-Changing Commits with Generated Code and Tests](https://mlsanity.com/paper/2607.13111/) (2026)
- [EA-Graph: Artifact-Anchored Verification Memory for Coding Agents under Upstream Drift](https://mlsanity.com/paper/2608.04278/) (2026)

## Abstract (arXiv)

Memory systems for coding agents must decide, when a repository changes, which of their stored claims have become false. Content anchoring invalidates a claim whenever the artifact it came from changes, which fires constantly. Semantic-equivalence classification asks whether a diff preserves behavior, a question about the diff rather than about any stored claim. We show the second signal fails for a reason unrelated to model capability: asked whether a commit preserves behavior, five models spanning a 40x price range fire on 59-72% of real commits and reach precisions of only 0.291 to 0.329 against a 0.25 base rate. Asked instead whether one specific claim still holds, the same models on the same diffs reach 0.705 to 0.974. A control that hands the behavior-preservation judge the claim text, changing only the question, moves precision by 0.010 and 0.016; changing the question moves it by 0.49 and 0.65. We also compare against pytest-testmon, a deployed regression-test selector with coverage-derived dependency data: it reaches 0.868 recall at 0.415 precision, so near-complete knowledge of what a change can reach does not identify what it falsifies. Ground truth is execution, not annotation: a claim is a test function passing at commit t, and it has flipped if that same assertion text fails at t+1. Building this required an observation we did not find in prior work. On a CI-gated mainline a commit that leaves a pre-existing test failing cannot merge, so the naive construction has an empty positive class by design. We report 10,369 claims with 184 execution-verified flips mined from 23 Python libraries, splits held out by repository, a post-knowledge-cutoff split, a shuffled-diff null, a paraphrase control, and a leave-one-repository-out analysis over 17 repositories.