mlsanityData to 7 Oct 2026

Embodied Agents Meet Personalization: Investigating Challenges and Solutions Through the Lens of Memory Utilization

arXiv 2505.16348 ↗First posted 22 May 2025cs.CLcited by 15

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

LLM-powered embodied agents have shown success on conventional object-rearrangement tasks, but providing personalized assistance that leverages user-specific knowledge from past interactions presents new challenges. We investigate these challenges through the lens of agents' memory utilization along two critical dimensions: object semantics (identifying objects based on personal meaning) and user patterns (recalling sequences from behavioral routines). To assess these capabilities, we construct MEMENTO, an end-to-end two-stage evaluation framework comprising single-memory and joint-memory tasks. Our experiments reveal that current agents can recall simple object semantics but struggle to apply sequential user patterns to planning. Through in-depth analysis, we identify two critical bottlenecks: information overload and coordination failures when handling multiple memories. Based on these findings, we explore memory architectural approaches to address these challenges. Given our observation that episodic memory provides both personalized knowledge and in-context learning benefits, we design a hierarchical knowledge graph-based user-profile memory module that separately manages personalized knowledge, achieving substantial improvements on both single and joint-memory tasks. Project website: https://connoriginal.github.io/MEMENTO

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 39 of its 72 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

39 of 72 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025A-MEM: Agentic Memory for LLM Agents17 Feb

Other references · 38

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  10. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  11. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  12. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
26 more
  1. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  2. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  3. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  4. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  5. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  6. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  7. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  8. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  9. 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 171
  10. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  11. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  12. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 131
  13. 2025A Comprehensive Survey on Long Context Language Modelingcited by 113
  14. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  15. 2024PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Taskscited by 78
  16. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 72
  17. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 71
  18. 2023Query Expansion by Prompting Large Language Modelscited by 56
  19. 2024Larimar: Large Language Models with Episodic Memory Controlcited by 53
  20. 2024Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generationcited by 44
  21. 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systemscited by 38
  22. 2023Open-Ended Instructable Embodied Agents with Memory-Augmented Large Language Modelscited by 37
  23. 2023Think, Act, and Ask: Open-World Interactive Personalized Robot Navigationcited by 31
  24. 2021My House, My Rules: Learning Tidying Preferences with Graph Neural Networkscited by 27
  25. 2025Episodic Memories Generation and Evaluation Benchmark for Large Language Modelscited by 14
  26. 2025Inferring the Unspoken: Aligning Embodied Agents with Implicit Preferencescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 10.

15 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  3. 2026Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitationscited by 20
  4. 2025Bring My Cup! Personalizing Vision-Language-Action Models with Visual Attentive Promptingcited by 5
  5. 2026PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraintscited by 5
  6. 2026FragFuse: Bypassing Access Control of Large Language Model Agents via Memory-Based Query Fragmentation and Fusioncited by 4
  7. 2026WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agentscited by 4
  8. 2025Designing Memory-Augmented AR Agents for Spatiotemporal Reasoning in Personalized Task Assistancecited by 2
  9. 2026Towards Direct Evaluation of Harness Optimizers via Priority Rankingcited by 2
  10. 2025Apples on the Table? Evaluating Text-Guided 3D Scene Synthesis via Fine-Grained Constraint Verificationcited by 1
  11. 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agentscited by 1
  12. 2026TaskGround: Structured Executable Task Inference for Full-Scene Household Reasoningcited by 0
3 more
  1. 2026Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactionscited by 0
  2. 2026FoundYou: A Unified Model for Personalized Segmentation and Retrievalcited by 0
  3. 2026PRAGMA: Evaluating Personalized Guidance with Memory Alignment in Lifelong Conversationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
  2. 2026EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks
  3. 2026Analytic Concept-Centric Memory for Agentic Embodied Manipulation
  4. 2026Graph-Based Personalized Memory for LLM Agents: Representation, Evolution, Retrieval, and Evaluation
  5. 2026eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents
  6. 2026Controllable Memory Usage: Balancing Anchoring and Innovation in Long-Term Human-Agent Interaction
  7. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agents
  8. 2026Muscle Memory for Agents: Compile not Merely Retrieve