mlsanityData to 7 Oct 2026

Temporal Validity on Real Software Histories: Eliminating Stale-Fact Errors in Code-Assistant Memory over GitHub Fixes

arXiv 2608.20685 ↗First posted 21 Aug 2026cs.SE cs.AI cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG for code

Abstract (arXiv)

Retrieval-augmented generation (RAG) has no model of time: when a fact changes across a coding session - a function is renamed, an endpoint moves, a dependency is bumped - RAG retrieves both the old and new value with near-identical similarity and cannot tell which is current, so it serves the superseded value. Paper 1 showed, on synthetic single-value benchmarks, that a deterministic (subject, relation, object) supersession memory eliminates this failure. Here we validate it end-to-end on real software history. From 707 real GitHub issues (SWE-bench Lite + Verified) we extract 130 clean atomic state transitions, a fix that changes one identifiable value from a pre-fix to a post-fix form, and render each marker-free (the stale and current statements differ only in the value). On this set, MemStrata reaches 0.91 answer accuracy versus RAG's 0.57-0.59; and, the structural result, when forced to answer RAG serves the superseded value 36-38% of the time (an LLM reranker does not help) while MemStrata drives this to ~0, at RAG retrieval latency (~2.1 s vs ~18 s for the reranker). We are explicit about scope: only ~18% of real fixes are clean atomic transitions; Paper 2 isolates the memory mechanism on that class, and extraction coverage of the remaining fixes is the orthogonal problem we defer to follow-on work. A real product bug surfaced and was fixed during the study (a case/punctuation-insensitive value comparison), with the moat property (deterministic-supersession accuracy on clean code mutations) preserved and verified.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 7 of its 7 references that are in the map. First posted on arXiv 21 Aug 2026.

Builds on

7 of 7 references.

References · 7

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  6. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  7. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Retrieval Hurts Code Completion: A Diagnostic Study of Stale Repository Context
  2. 2026MemTrace: State-Consistent Memory for Long-Horizon Coding Agents
  3. 2024A Methodology for Evaluating RAG Systems: A Case Study On Configuration Dependency Validation
  4. 2026VibeMemBench: Evaluating Memory Systems for Coding Agents on Real Repository Coding Tasks
  5. 2026Not All RAGs Are Created Equal: A Component-Wise Empirical Study for Software Engineering Tasks
  6. 2026NuggetIndex: Governed Atomic Retrieval for Maintainable RAG
  7. 2026AtomicCommitBench: Can Coding Agents Reconstruct Commit Histories from Squashed Patches?
  8. 2026Beyond "What to Retrieve": Uncertainty in Retrieval-Augmented Code Generation