# PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents

arXiv: https://arxiv.org/abs/2603.03296 · HTML: https://mlsanity.com/paper/2603.03296/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 74 references that are in the map.
- First posted on arXiv 6 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 21 of 21)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://mlsanity.com/paper/2407.18901/) (2024)
- [∞Bench: Extending Long Context Evaluation Beyond 100K Tokens](https://mlsanity.com/paper/2402.13718/) (2024)
- [From RAG to Memory: Non-Parametric Continual Learning for Large Language Models](https://mlsanity.com/paper/2502.14802/) (2025)
- [Multimodal Web Navigation with Instruction-Finetuned Foundation Models](https://mlsanity.com/paper/2305.11854/) (2023)
- [A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts](https://mlsanity.com/paper/2402.09727/) (2024)
- [AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents](https://mlsanity.com/paper/2410.13825/) (2024)
- [Memp: Exploring Agent Procedural Memory](https://mlsanity.com/paper/2508.06433/) (2025)
- [AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents](https://mlsanity.com/paper/2407.04363/) (2024)
- [SteP: Stacked LLM Policies for Web Actions](https://mlsanity.com/paper/2310.03720/) (2023)
- [LiCoMemory: Lightweight and Cognitive Agentic Memory for Efficient Long-Term Reasoning](https://mlsanity.com/paper/2511.01448/) (2025)
- [PropRAG: Guiding Retrieval with Beam Search over Proposition Paths](https://mlsanity.com/paper/2504.18070/) (2025)

## Built on by (most cited first, 30 of 32)

- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [WebXSkill: Skill Learning for Autonomous Web Agents](https://mlsanity.com/paper/2604.13318/) (2026)
- [A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications](https://mlsanity.com/paper/2605.07358/) (2026)
- [Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents](https://mlsanity.com/paper/2605.20616/) (2026)
- [Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems](https://mlsanity.com/paper/2605.26302/) (2026)
- [HyperSkill: Self-Evolving LLM Agents via Hypergraph-Structured Skill Memory](https://mlsanity.com/paper/2608.16114/) (2026)
- [MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems](https://mlsanity.com/paper/2605.28732/) (2026)
- [Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents](https://mlsanity.com/paper/2606.15017/) (2026)
- [Demystifying Agent Skills: Why They Work-Until They Don't](https://mlsanity.com/paper/2608.14036/) (2026)
- [User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction](https://mlsanity.com/paper/2603.20939/) (2026)
- [Mem-π: Adaptive Memory through Learning When and What to Generate](https://mlsanity.com/paper/2605.21463/) (2026)
- [Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents](https://mlsanity.com/paper/2605.25535/) (2026)
- [LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception](https://mlsanity.com/paper/2606.30005/) (2026)
- [MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents](https://mlsanity.com/paper/2607.24097/) (2026)
- [Retrieval-Driven Memory Reconsolidation for Long-Term LLM Agents](https://mlsanity.com/paper/2609.16053/) (2026)
- [Rethinking Memory as Continuously Evolving Connectivity](https://mlsanity.com/paper/2605.28773/) (2026)
- [Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline](https://mlsanity.com/paper/2606.04315/) (2026)
- [MESA:Task-Adaptive Multi-Structure Evidence Selection for Long-Horizon Agent Memory](https://mlsanity.com/paper/2608.10108/) (2026)
- [MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use](https://mlsanity.com/paper/2608.20202/) (2026)
- [StudentSim: Training LLM-based Student Simulators](https://mlsanity.com/paper/2609.01591/) (2026)
- [Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents](https://mlsanity.com/paper/2609.11060/) (2026)
- [CoMemBench: Benchmarking Collaborative Memory Boundaries across Multi-Agent Workflow Topologies](https://mlsanity.com/paper/2609.32192/) (2026)
- [CommitDistill: A Lightweight Knowledge-Centric Memory Layer for Software Repositories](https://mlsanity.com/paper/2605.18284/) (2026)
- [PraMem: Practice-derived Experiential Memory for Long-horizon Behavior Prediction](https://mlsanity.com/paper/2607.02881/) (2026)
- [VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement](https://mlsanity.com/paper/2609.03153/) (2026)
- [CreaMem: A Scene-Aware Memory Architecture for Personalized Agents](https://mlsanity.com/paper/2609.08550/) (2026)
- [How Strongly Should Task State Influence an LLM Agent?](https://mlsanity.com/paper/2609.25686/) (2026)
- [EnSIMem: Entity-Structured Indexing for Long-Term Agent Memory](https://mlsanity.com/paper/2609.27279/) (2026)
- [Topological Coherence for Self-evolving Multi-agent Systems](https://mlsanity.com/paper/2609.37953/) (2026)
- [MemCo: Memory-Centric Collaboration for Generalizing LLM Agents to Unseen Environments](https://mlsanity.com/paper/2610.07376/) (2026)

## Abstract (arXiv)

Long-term memory is essential for large language model (LLM) agents operating in complex environments, yet existing memory designs are either task-specific and non-transferable, or task-agnostic but less effective due to low task-relevance and context explosion from raw memory retrieval. We propose PlugMem, a task-agnostic plugin memory module that can be attached to arbitrary LLM agents without task-specific redesign. Motivated by the fact that decision-relevant information is concentrated as abstract knowledge rather than raw experience, we draw on cognitive science to structure episodic memories into a compact, extensible knowledge-centric memory graph that explicitly represents propositional and prescriptive knowledge. This representation enables efficient memory retrieval and reasoning over task-relevant knowledge, rather than verbose raw trajectories, and departs from other graph-based methods like GraphRAG by treating knowledge as the unit of memory access and organization instead of entities or text chunks. We evaluate PlugMem unchanged across three heterogeneous benchmarks (long-horizon conversational question answering, multi-hop knowledge retrieval, and web agent tasks). The results show that PlugMem consistently outperforms task-agnostic baselines and exceeds task-specific memory designs, while also achieving the highest information density under a unified information-theoretic analysis. Code and data are available at https://github.com/TIMAN-group/PlugMem.