Communication-Efficient Desire Alignment for Embodied Agent-Human Adaptation
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
While embodied agents have made significant progress in performing complex physical tasks, real-world applications demand more than pure task execution. The agents must collaborate with unfamiliar agents and human users, whose goals are often vague and implicit. In such settings, interpreting ambiguous instructions and uncovering underlying desires is essential for effective assistance. Therefore, fast and accurate desire alignment becomes a critical capability for embodied agents. In this work, we first develop a home assistance simulation environment HA-Desire that integrates an LLM-driven proxy human user exhibiting realistic value-driven goal selection and communication. The ego agent must interact with this proxy user to infer and adapt to the user's latent desires. To achieve this, we present a novel framework FAMER for fast desire alignment, which introduces a desire-based mental reasoning mechanism to identify user intent and filter desire-irrelevant actions. We further design a reflection-based communication module that reduces redundant inquiries, and incorporate goal-relevant information extraction with memory persistence to improve information reuse and reduce unnecessary exploration. Extensive experiments demonstrate that our framework significantly enhances both task execution and communication efficiency, enabling embodied agents to quickly adapt to user-specific desires in complex embodied environments.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 35 of its 48 references that are in the map. First posted on arXiv 28 May 2025.
Builds on
35 of 48 references.
Closest ancestors · 4
Same direction, newest first
- 2024CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation7 Nov
- 2023ProAgent: Building Proactive Cooperative Agents with Large Language Models22 Aug
- 2023Building Cooperative Embodied Agents Modularly with Large Language Models5 Jul
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar
Other references · 31
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2017Mask R-CNNcited by 7,277
- 2024GPT-4o System Cardcited by 4,588
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
19 more
- 2018VirtualHome: Simulating Household Activities via Programscited by 415
- 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
- 2023AI Alignment: A Comprehensive Surveycited by 263
- 2018Machine Theory of Mindcited by 226
- 2020"Other-Play" for Zero-Shot Coordinationcited by 198
- 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
- 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 183
- 2021Collaborating with Humans without Human Datacited by 142
- 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 133
- 2021Off-Belief Learningcited by 75
- 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 58
- 2015A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systemscited by 44
- 2020Towards Open Ad Hoc Teamwork Using Graph-based Policy Learningcited by 37
- 2017Pragmatic-Pedagogic Value Alignmentcited by 36
- 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 32
- 2024Simulating Human-like Daily Activities with Desire-driven Autonomycited by 30
- 2024Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challengecited by 13
- 2025AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learningcited by 12
- 2024Fast Peer Adaptation with Context-aware Explorationcited by 7
Built on by
Citing papers per year: 2025: 1; 2026: 3.
4 papers.
- 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
- 2025MindPower: Enabling Theory-of-Mind Reasoning in VLM-based Embodied Agentscited by 5
- 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulationcited by 2
- 2026RescueBench: Can Embodied Agents Save Lives in the Wild ?cited by 0
- 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation15 Jul 2026
- 2026RescueBench: Can Embodied Agents Save Lives in the Wild ?1 Jun 2026
- 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents18 Jan 2026
- 2025MindPower: Enabling Theory-of-Mind Reasoning in VLM-based Embodied Agents28 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2024GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment
- 2026Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue
- 2024Goal Inference from Open-Ended Dialog
- 2026ELITE: Experiential Learning and Intent-Aware Transfer for Self-improving Embodied Agents
- 2023Asking Before Acting: Gather Information in Embodied Decision Making with Language Models
- 2026ETA: A New Agentic Paradigm for Embodied Tasks
- 2023InterAct: Exploring the Potentials of ChatGPT as a Cooperative Agent
- 2025Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability