mlsanityData to 14 Oct 2026

Learning Hierarchical Procedural Memory for LLM Agents through Bayesian Selection and Contrastive Refinement

arXiv 2512.18950 ↗First posted 22 Dec 2025cs.LG cs.AIcited by 20

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

We present MACLA, a framework that decouples reasoning from learning by maintaining a frozen large language model while performing all adaptation in an external hierarchical procedural memory. MACLA extracts reusable procedures from trajectories, tracks reliability via Bayesian posteriors, selects actions through expected-utility scoring, and refines procedures by contrasting successes and failures. Across four benchmarks (ALFWorld, WebShop, TravelPlanner, InterCodeSQL), MACLA achieves 78.1 percent average performance, outperforming all baselines. On ALFWorld unseen tasks, MACLA reaches 90.3 percent with 3.1 percent positive generalization. The system constructs memory in 56 seconds, 2800 times faster than the state-of-the-art LLM parameter-training baseline, compressing 2851 trajectories into 187 procedures. Experimental results demonstrate that structured external memory with Bayesian selection and contrastive refinement enables sample-efficient, interpretable, and continually improving agents without LLM parameter updates.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 16 of its 28 references that are in the map. First posted on arXiv 22 Dec 2025.

Builds on

16 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agents21 Apr

Other references · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  8. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392
  9. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 338
  10. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  11. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
  12. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
3 more
  1. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 197
  2. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinementcited by 88
  3. 2024AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learningcited by 61

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 20.

20 papers.

Sort:

By area:

  1. 2026MARS: Modular Agent with Reflective Search for Automated AI Researchcited by 25
  2. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 21
  3. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 14
  4. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 9
  5. 2026SuperLocalMemory: Privacy-Preserving Multi-Agent Memory with Bayesian Trust Defense Against Memory Poisoningcited by 7
  6. 2026Ratchet: How Reliable Must an LLM Judge Be to Retire a Skill?cited by 5
  7. 2026Oblivion: Self-Adaptive Agentic Memory Control through Decay-Driven Activationcited by 3
  8. 2026View-oriented Conversation Compiler for Agent Trace Analysiscited by 2
  9. 2026AdaptFuse: Training-Free Sequential Preference Inference via Externalized Bayesian Computationcited by 2
  10. 2026Web2BigTable: A Bi-Level Multi-Agent LLM System for Internet-Scale Information Search and Extractioncited by 2
  11. 2026AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agentscited by 2
  12. 2026Skill-CMIB: Multimodal Agent Skill for Consistent Action via Conditional Multimodal Information Bottleneckcited by 1
8 more
  1. 2026AgentSpec: Understanding Embodied Agent Scaffolds Through Controlled Compositioncited by 1
  2. 2026Making AI Scientists Auditable from Evidence to Claimcited by 1
  3. 2026Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Designcited by 1
  4. 2026CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflectioncited by 0
  5. 2026Task-Adaptive Retrieval over Agentic Multi-Modal Web Histories via Learned Graph Memorycited by 0
  6. 2026Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMscited by 0
  7. 2026Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replaycited by 0
  8. 2026AECG: Asymmetric Experience Consolidation and Governance In Multi-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents
  2. 2025From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory
  3. 2026MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning
  4. 2026AdMem: Advanced Memory for Task-solving Agents
  5. 2025H2R: Hierarchical Hindsight Reflection for Multi-Task LLM Agents
  6. 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
  7. 2026SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent
  8. 2026MemGym: a Long-Horizon Memory Environment for LLM Agents