mlsanityData to 14 Oct 2026

Communication-Efficient Desire Alignment for Embodied Agent-Human Adaptation

arXiv 2505.22503 ↗First posted 28 May 2025cs.RO cs.AI cs.MAcited by 4

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

While embodied agents have made significant progress in performing complex physical tasks, real-world applications demand more than pure task execution. The agents must collaborate with unfamiliar agents and human users, whose goals are often vague and implicit. In such settings, interpreting ambiguous instructions and uncovering underlying desires is essential for effective assistance. Therefore, fast and accurate desire alignment becomes a critical capability for embodied agents. In this work, we first develop a home assistance simulation environment HA-Desire that integrates an LLM-driven proxy human user exhibiting realistic value-driven goal selection and communication. The ego agent must interact with this proxy user to infer and adapt to the user's latent desires. To achieve this, we present a novel framework FAMER for fast desire alignment, which introduces a desire-based mental reasoning mechanism to identify user intent and filter desire-irrelevant actions. We further design a reflection-based communication module that reduces redundant inquiries, and incorporate goal-relevant information extraction with memory persistence to improve information reuse and reduce unnecessary exploration. Extensive experiments demonstrate that our framework significantly enhances both task execution and communication efficiency, enabling embodied agents to quickly adapt to user-specific desires in complex embodied environments.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 35 of its 48 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

35 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation7 Nov
  2. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Models22 Aug
  3. 2023Building Cooperative Embodied Agents Modularly with Large Language Models5 Jul
  4. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2017Mask R-CNNcited by 7,277
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  8. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  11. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  12. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
19 more
  1. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  2. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  3. 2023AI Alignment: A Comprehensive Surveycited by 263
  4. 2018Machine Theory of Mindcited by 226
  5. 2020"Other-Play" for Zero-Shot Coordinationcited by 198
  6. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
  7. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 183
  8. 2021Collaborating with Humans without Human Datacited by 142
  9. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 133
  10. 2021Off-Belief Learningcited by 75
  11. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 58
  12. 2015A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systemscited by 44
  13. 2020Towards Open Ad Hoc Teamwork Using Graph-based Policy Learningcited by 37
  14. 2017Pragmatic-Pedagogic Value Alignmentcited by 36
  15. 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 32
  16. 2024Simulating Human-like Daily Activities with Desire-driven Autonomycited by 30
  17. 2024Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challengecited by 13
  18. 2025AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learningcited by 12
  19. 2024Fast Peer Adaptation with Context-aware Explorationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment
  2. 2026Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue
  3. 2024Goal Inference from Open-Ended Dialog
  4. 2026ELITE: Experiential Learning and Intent-Aware Transfer for Self-improving Embodied Agents
  5. 2023Asking Before Acting: Gather Information in Embodied Decision Making with Language Models
  6. 2026ETA: A New Agentic Paradigm for Embodied Tasks
  7. 2023InterAct: Exploring the Potentials of ChatGPT as a Cooperative Agent
  8. 2025Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability