mlsanityData to 7 Oct 2026

Inferring the Goals of Communicating Agents from Actions and Instructions

arXiv 2306.16207 ↗First posted 28 Jun 2023cs.AI cs.CL cs.ROcited by 17

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

When humans cooperate, they frequently coordinate their activity through both verbal communication and non-verbal actions, using this information to infer a shared goal and plan. How can we model this inferential ability? In this paper, we introduce a model of a cooperative team where one agent, the principal, may communicate natural language instructions about their shared plan to another agent, the assistant, using GPT-3 as a likelihood function for instruction utterances. We then show how a third person observer can infer the team's goal via multi-modal Bayesian inverse planning from actions and instructions, computing the posterior distribution over goals under the assumption that agents will act and communicate rationally to achieve them. We evaluate this approach by comparing it with human goal inferences in a multi-agent gridworld, finding that our model's inferences closely correlate with human judgments (R = 0.96). When compared to inference from actions alone, we also find that instructions lead to more rapid and less uncertain goal inference, highlighting the importance of verbal communication for cooperative agents.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 20 of its 45 references that are in the map. First posted on arXiv 28 Jun 2023.

Builds on

20 of 45 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  4. 2016Cooperative Inverse Reinforcement Learningcited by 298
  5. 2023Reward Design with Language Modelscited by 257
  6. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 119
  7. 2022Language Model Cascadescited by 104
  8. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  9. 2020Online Bayesian Goal Inference for Boundedly-Rational Planning Agentscited by 67
  10. 2019On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inferencecited by 49
  11. 2017Pragmatic-Pedagogic Value Alignmentcited by 36
  12. 2019Theory of Minds: Understanding Behavior in Groups Through Inverse Planningcited by 30
8 more
  1. 2017Should Robots be Obedient?cited by 20
  2. 2021Modeling the Mistakes of Boundedly Rational Agents Within a Bayesian Theory of Mindcited by 19
  3. 2018Reward Learning from Narrated Demonstrationscited by 13
  4. 2023LaMPP: Language Models as Probabilistic Priors for Perception and Actioncited by 11
  5. 2018Nested Reasoning About Autonomous Agents Using Probabilistic Programscited by 7
  6. 2021Modeling Communication to Coordinate Perspectives in Cooperationcited by 5
  7. 2017Probabilistic programs for inferring the goals of autonomous agentscited by 4
  8. 2020Learning a metacognition for object perceptioncited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 8; 2025: 7.

17 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  3. 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
  4. 2024Large Language Models Assume People are More Rational than We Really arecited by 49
  5. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  6. 2025On Benchmarking Human-Like Intelligence in Machinescited by 24
  7. 2024GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignmentcited by 16
  8. 2023Guide Your Agent with Adaptive Multimodal Rewardscited by 10
  9. 2024Grounding Language about Belief in a Bayesian Theory-of-Mindcited by 8
  10. 2024Infinite Ends from Finite Samples: Open-Ended Goal Inference as Top-Down Bayesian Filtering of Bottom-Up Proposalscited by 6
  11. 2024Understanding Epistemic Language with a Language-augmented Bayesian Theory of Mindcited by 6
  12. 2025Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Flycited by 6
5 more
  1. 2024Goal Inference from Open-Ended Dialogcited by 4
  2. 2025Gricean Norms as a Basis for Effective Collaborationcited by 3
  3. 2025Open-Ended Goal Inference through Actions and Language for Human-Robot Collaborationcited by 3
  4. 2023Utility-based Adaptive Teaching Strategies using Bayesian Theory of Mindcited by 1
  5. 2025Language and Experience: A Computational Model of Social Learning in Complex Taskscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Theory of Mind in Action: The Instruction Inference Task in Dynamic Human-Agent Collaboration
  2. 2024Infer Human's Intentions Before Following Natural Language Instructions
  3. 2026Inferring Latent Intentions: Attributional Natural Language Inference in LLM Agents
  4. 2025ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback
  5. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation
  6. 2024Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models
  7. 2026Belief Without Behavior: Measuring the Translation of Theory of Mind into Coordinated Social Action in Vision-Language Models
  8. 2026A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents