Impact Is Not Invalidation: Ask About the Claim, Not the Diff
On the map: LLM agents and tool use → Agent memory systems also fits: Software engineering coding agents
Abstract (arXiv)
Memory systems for coding agents must decide, when a repository changes, which of their stored claims have become false. Content anchoring invalidates a claim whenever the artifact it came from changes, which fires constantly. Semantic-equivalence classification asks whether a diff preserves behavior, a question about the diff rather than about any stored claim. We show the second signal fails for a reason unrelated to model capability: asked whether a commit preserves behavior, five models spanning a 40x price range fire on 59-72% of real commits and reach precisions of only 0.291 to 0.329 against a 0.25 base rate. Asked instead whether one specific claim still holds, the same models on the same diffs reach 0.705 to 0.974. A control that hands the behavior-preservation judge the claim text, changing only the question, moves precision by 0.010 and 0.016; changing the question moves it by 0.49 and 0.65. We also compare against pytest-testmon, a deployed regression-test selector with coverage-derived dependency data: it reaches 0.868 recall at 0.415 precision, so near-complete knowledge of what a change can reach does not identify what it falsifies. Ground truth is execution, not annotation: a claim is a test function passing at commit t, and it has flipped if that same assertion text fails at t+1. Building this required an observation we did not find in prior work. On a CI-gated mainline a commit that leaves a pre-existing test failing cannot merge, so the naive construction has an empty positive class by design. We report 10,369 claims with 184 execution-verified flips mined from 23 Python libraries, splits held out by repository, a post-knowledge-cutoff split, a shuffled-diff null, a paraphrase control, and a leave-one-repository-out analysis over 17 repositories.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 6 of its 13 references that are in the map. First posted on arXiv 20 Sep 2026.
Builds on
6 of 13 references.
Closest ancestors · 2
Same direction, newest first
- 2026EA-Graph: Artifact-Anchored Verification Memory for Coding Agents under Upstream Drift4 Aug
- 2026Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge25 Jun
Other references · 4
- 2021Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Reviewcited by 12
- 2024ChangeGuard: Validating Code Changes via Pairwise Learning-Guided Executioncited by 6
- 2026Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrievalcited by 3
- 2026SemaDiff: Identifying Semantic-Changing Commits with Generated Code and Testscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026PACMI: Provenance-Aware Cascading Memory Invalidation for Long-Term LLM Agents
- 2026When Memory Updates but Behavior Does Not: Repairing Implicit Stale Dependencies in Personalized Agent Responses
- 2026When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory
- 2026MemTrace: State-Consistent Memory for Long-Horizon Coding Agents
- 2026Change-Provenant Supervision: Governing Learned Artifacts Under Policy Change ↗
- 2026Correct Now, Insufficient Later: Auditing Update Sufficiency in Context Compression
- 2026From Faulty Memories to Corrected Actions: Dependency-Guided Rollback Repair for Memory-Augmented Agents
- 2026Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability