mlsanityData to 14 Oct 2026

PersONAL: Towards a Comprehensive Benchmark for Personalized Embodied Agents

arXiv 2509.19843 ↗First posted 24 Sep 2025cs.CV cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent advances in Embodied AI have enabled agents to perform increasingly complex tasks and adapt to diverse environments. However, deploying such agents in realistic human-centered scenarios, such as domestic households, remains challenging, particularly due to the difficulty of modeling individual human preferences and behaviors. In this work, we introduce PersONAL (PERSonalized Object Navigation And Localization, a comprehensive benchmark designed to study personalization in Embodied AI. Agents must identify, retrieve, and navigate to objects associated with specific users, responding to natural-language queries such as "find Lily's backpack". PersONAL comprises over 2,000 high-quality episodes across 30+ photorealistic homes from the HM3D dataset. Each episode includes a natural-language scene description with explicit associations between objects and their owners, requiring agents to reason over user-specific semantics. The benchmark supports two evaluation modes: (1) active navigation in unseen environments, and (2) object grounding in previously mapped scenes. Experiments with state-of-the-art baselines reveal a substantial gap to human performance, highlighting the need for embodied agents capable of perceiving, reasoning, and memorizing over personalized information; paving the way towards real-world assistive robot.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

38 of 43 references.

References · 38

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  5. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  8. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  9. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  10. 2017Learning multiple visual domains with residual adapterscited by 557
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  12. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
26 more
  1. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  2. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  3. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
  4. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  5. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 193
  6. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  7. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  8. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  9. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
  10. 2023GOAT: GO to Any Thingcited by 89
  11. 20243D-Mem: 3D Scene Memory for Embodied Exploration and Reasoningcited by 75
  12. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  13. 2025Embodied AI Agents: Modeling the Worldcited by 61
  14. 2024MyVLM: Personalizing VLMs for User-Specific Queriescited by 58
  15. 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 55
  16. 2023Navigating to Objects Specified by Imagescited by 51
  17. 2024VLM See, Robot Do: Human Demo Video to Robot Action Plan via Vision Language Modelcited by 37
  18. 2023Think, Act, and Ask: Open-World Interactive Personalized Robot Navigationcited by 31
  19. 2024TANGO: Training-free Embodied AI Agents for Open-world Taskscited by 27
  20. 2023Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferencescited by 26
  21. 2024An Interactive Agent Foundation Modelcited by 25
  22. 2024One Map to Find Them All: Real-time Open-Vocabulary Mapping for Zero-shot Multi-Object Navigationcited by 13
  23. 2024Personalized Large Vision-Language Modelscited by 12
  24. 2025FindingDory: A Benchmark to Evaluate Memory in Embodied Agentscited by 12
  25. 2024Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environmentscited by 11
  26. 2024TINA: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 14
  2. 2026IntentionNav: A Benchmark for Intent-Driven Object Navigation from Implicit Human Instructioncited by 1
  3. 2026Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactionscited by 0
  4. 2026EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Taskscited by 0