Learning When to Remember: Risk-Sensitive Contextual Bandits for Abstention-Aware Memory Retrieval in LLM-Based Coding Agents
On the map: LLM agents and tool use → Agent memory systems
Abstract (arXiv)
Large language model (LLM)-based coding agents increasingly rely on external memory to reuse prior debugging experience, repair traces, and repository-local operational knowledge. However, retrieved memory is useful only when the current failure is genuinely compatible with a previous one; superficial similarity in stack traces, terminal errors, paths, or configuration symptoms can lead to unsafe memory injection. This paper reframes issue-memory use as a selective, risk-sensitive control problem rather than a pure top-k retrieval problem. We introduce RSCB-MC, a risk-sensitive contextual bandit memory controller that decides whether an agent should use no memory, inject the top resolution, summarize multiple candidates, perform high-precision or high-recall retrieval, abstain, or ask for feedback. The system stores reusable issue knowledge through a pattern-variant-episode schema and converts retrieval evidence into a fixed 16-feature contextual state capturing relevance, uncertainty, structural compatibility, feedback history, false-positive risk, latency, and token cost. Its reward design penalizes false-positive memory injection more strongly than missed reuse, making non-injection and abstention first-class safety actions. In deterministic smoke-scale artifacts, RSCB-MC obtains the strongest non-oracle offline replay success rate, 62.5%, while maintaining a 0.0% false-positive rate. In a bounded 200-case hot-path validation, it reaches 60.5% proxy success with 0.0% false positives and a 331.466 microseconds p95 decision latency. The results show that, for coding-agent memory, the key question is not only which memory is most similar, but whether any retrieved memory is safe enough to influence the debugging trajectory.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 26 of its 31 references that are in the map. First posted on arXiv 30 Apr 2026.
Builds on
26 of 31 references.
Closest ancestors · 6
Same direction, newest first
- 2026ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents2 Apr
- 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers8 Mar
- 2026Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents5 Jan
- 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning27 Aug
- 2025MIRIX: Multi-Agent Memory System for LLM-Based Agents10 Jul
- 2023Empowering Working Memory for Large Language Model Agents22 Dec
Other references · 20
- 2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 68
- 2023KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Basescited by 52
- 2023RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repaircited by 43
- 2025Parametric Retrieval Augmented Generationcited by 38
- 2023Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanismcited by 33
- 2025Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Modelscited by 20
- 2025Supervising the search process produces reliable and generalizable information-seeking agentscited by 20
- 2026Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learnedcited by 18
- 2023Selective Generation for Controllable Language Modelscited by 15
- 2024Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guaranteescited by 15
- 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
- 2025A Contemporary Survey of Large Language Model Assisted Program Analysiscited by 10
8 more
- 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Modelscited by 9
- 2025Repair Ingredients Are All You Need: Improving Large Language Model-Based Program Repair via Repair Ingredients Searchcited by 8
- 2024Reliable Decision Making via Calibration Oriented Retrieval Augmented Generationcited by 6
- 2025Agentic Software Issue Resolution with Large Language Models: A Surveycited by 6
- 2026TreePS-RAG: Tree-based Process Supervision for Reinforcement Learning in Agentic RAGcited by 5
- 2025FLAIR: Feedback Learning for Adaptive Information Retrievalcited by 4
- 2026I-CALM: Incentivizing Confidence-Aware Abstention for LLM Selective Answeringcited by 4
- 2025SIRAG: Towards Stable and Interpretable RAG with A Process-Supervised Multi-Agent Frameworkcited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Memory Reward Inflation in Self-Improving LLM Agentscited by 3
- 2026From Relevance to Execution Utility: Reward-Aware Dynamic Execution Gating for Skill-Based LLM Agentscited by 2
- 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agentscited by 0
- 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents30 Sep 2026
- 2026From Relevance to Execution Utility: Reward-Aware Dynamic Execution Gating for Skill-Based LLM Agents10 Aug 2026
- 2026Memory Reward Inflation in Self-Improving LLM Agents29 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026SF-AMS: Strategic Forgetting for Structured Memory in LLM Agent
- 2026Decision-Aware Memory Cards: Counterfactual-Inspired Context Selection and Compression for Tool-Using LLM Agents
- 2026Feedback-Normalized Developer Memory for Reinforcement-Learning Coding Agents: A Safety-Gated MCP Architecture
- 2026MeClear: Cooperative Game-Theoretic Attribution and Risk-Aware Memory Clearance for Long-Horizon LLM Agents
- 2026Memory as Infrastructure: Reliability Engineering for Persistent Agent Memory in Months-Long LLM-Assisted Development
- 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
- 2026UpliftMem: Learning Set-Level Uplift for Agent Memory Retrieval
- 2026Choosing How to Remember: Adaptive Memory Structures for LLM Agents