# PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments

arXiv: https://arxiv.org/abs/2603.23231 · HTML: https://mlsanity.com/paper/2603.23231/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 71 of its 87 references that are in the map.
- First posted on arXiv 24 Mar 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)
- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 30 of 71)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security](https://mlsanity.com/paper/2401.05459/) (2024)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning](https://mlsanity.com/paper/2508.19828/) (2025)
- [Query Rewriting for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2305.14283/) (2023)
- [LightMem: Lightweight and Efficient Memory-Augmented Generation](https://mlsanity.com/paper/2510.18866/) (2025)
- [Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale](https://mlsanity.com/paper/2504.14225/) (2025)
- [MIRIX: Multi-Agent Memory System for LLM-Based Agents](https://mlsanity.com/paper/2507.07957/) (2025)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models](https://mlsanity.com/paper/2404.16019/) (2024)
- [MemOS: A Memory OS for AI System](https://mlsanity.com/paper/2507.03724/) (2025)
- [HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly](https://mlsanity.com/paper/2410.02694/) (2024)

## Built on by (most cited first, 14 of 14)

- [Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems](https://mlsanity.com/paper/2605.26302/) (2026)
- [π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows](https://mlsanity.com/paper/2605.14678/) (2026)
- [MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery](https://mlsanity.com/paper/2606.24595/) (2026)
- [TriggerBench: Investigating Prospective Memory for Large Language Models](https://mlsanity.com/paper/2606.23459/) (2026)
- [VehicleMemBench: An Executable Benchmark for Multi-User Long-Term Memory in In-Vehicle Agents](https://mlsanity.com/paper/2603.23840/) (2026)
- [AtomMem: Building Simple and Effective Memory System for LLM Agents via Atomic Facts](https://mlsanity.com/paper/2606.19847/) (2026)
- [Learning What to Share and What to Personalize: Hierarchical Strategy Co-Evolution for Agent Memory](https://mlsanity.com/paper/2608.25329/) (2026)
- [The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads](https://mlsanity.com/paper/2608.04570/) (2026)
- [RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agents](https://mlsanity.com/paper/2609.23466/) (2026)
- [Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data](https://mlsanity.com/paper/2604.20261/) (2026)
- [Controlled Memory Interference in Continual LLM Agents](https://mlsanity.com/paper/2608.07622/) (2026)
- [Incorporating Cognitive Load and Knowledge Transfer for Multi-Domain Knowledge Tracing](https://mlsanity.com/paper/2608.24005/) (2026)
- [Memory Type Varies: Empowering LLM Agents for Long-Term Memory with Diverse Strategies](https://mlsanity.com/paper/2610.11573/) (2026)
- [DPPM: Dual-Path Parametric Memory for Personalized Language Models](https://mlsanity.com/paper/2610.11776/) (2026)

## Abstract (arXiv)

Empowering large language models with long-term memory is crucial for building agents that adapt to users' evolving needs. Existing evaluations of this capability typically interleave preference-related dialogues with irrelevant conversations, reducing the task to needle-in-a-haystack retrieval while ignoring relationships between events driving user preference evolution. Such settings overlook a fundamental characteristic of real-world personalization: preferences emerge gradually and accumulate across interactions within noisy contexts. To bridge this gap, we introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall. Additionally, we incorporate (1) text variability and (2) linguistic alignment to simulate erratic user inputs and individual idiolects in real-world data. PERMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time. We design both multiple-choice and interactive tasks to probe the model's understanding of persona along the interaction timeline. Experiments demonstrate that by linking related interactions, advanced memory systems extract precise preferences and reduce token consumption, outperforming traditional semantic retrieval of raw dialogues. Nevertheless, they still struggle to maintain a coherent persona across temporal depth and cross-domain interference, highlighting the need for more robust personalized memory management in agents. Our code and data are open-sourced at https://github.com/PolarisLiu1/PERMA.