WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agents
Not on the map. Why?
Abstract (arXiv)
To assist humans over extended periods in real homes, embodied agents must remember user routines, world states, and past interactions. Existing long-term memory benchmarks mainly evaluate language-centric retrieval and question answering, while embodied benchmarks often focus on short-horizon task execution without testing long-term memory use in dynamic environments. We introduce WorldLines, a project-driven benchmark for long-horizon embodied household assistance. It constructs temporally extended household traces with dialogues, actions, execution feedback, object and device state changes, and converts them into evidence-linked samples for Memory QA and Embodied Task Planning. We further propose ObsMem, an observer-grounded memory framework that maintains visibility-aware memories and action-native state trails for state-aware decisions. Experiments reveal persistent challenges in partial observability, overwritten world states, and translating long-term memory into embodied plans, while ObsMem offers a stronger reference architecture for this setting.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 17 Jun 2026.
Builds on
23 of 26 references.
References · 23
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
- 2025Memory in the Age of AI Agentscited by 303
- 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 294
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
11 more
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 221
- 2025Memory OS of AI Agentcited by 150
- 2024PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Taskscited by 82
- 2025HaluMem: Evaluating Hallucinations in Memory Systems of Agentscited by 54
- 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systemscited by 39
- 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environmentscited by 32
- 2026RealMem: Benchmarking LLMs in Real-World Memory-Driven Interactioncited by 20
- 2025Embodied Agents Meet Personalization: Investigating Challenges and Solutions Through the Lens of Memory Utilizationcited by 19
- 2026StructMem: Structured Memory for Long-Horizon Behavior in LLMscited by 18
- 2026Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memorycited by 10
- 2025EvoEmpirBench: Dynamic Spatial Reasoning with Agent-ExpVercited by 5
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot Learningcited by 3
- 2026No Place to Hide: Benchmarking Video Hallucination with Background-Controlled Pairscited by 2
- 2026In-Context Learning for Robots: Methods and Applicationscited by 1
- 2026EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Taskscited by 0
- 2026Are Frontier VLM Agents Ready to Be Robot Generalists? An Empirical Study with the Embodied Agent Arenacited by 0
- 2026Are Frontier VLM Agents Ready to Be Robot Generalists? An Empirical Study with the Embodied Agent Arena1 Oct 2026
- 2026In-Context Learning for Robots: Methods and Applications28 Sep 2026
- 2026RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot Learning24 Sep 2026
- 2026EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks23 Sep 2026
- 2026No Place to Hide: Benchmarking Video Hallucination with Background-Controlled Pairs30 Jun 2026