# Challenges in Human-Agent Communication arXiv: https://arxiv.org/abs/2412.10380 ยท HTML: https://mlsanity.com/paper/2412.10380/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. - Builds on 36 of its 85 references that are in the map. - First posted on arXiv 28 Nov 2024. ## Placed in - LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/) ## Builds on (most cited first, 30 of 36) - [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023) - [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023) - [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023) - [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022) - [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022) - [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023) - [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016) - [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023) - [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017) - [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018) - [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024) - [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022) - [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016) - [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024) - [On the Planning Abilities of Large Language Models : A Critical Investigation](https://mlsanity.com/paper/2305.15771/) (2023) - [AI Alignment: A Comprehensive Survey](https://mlsanity.com/paper/2310.19852/) (2023) - [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018) - [Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks](https://mlsanity.com/paper/2411.04468/) (2024) - [Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy](https://mlsanity.com/paper/2002.04087/) (2020) - [Harms from Increasingly Agentic Algorithmic Systems](https://mlsanity.com/paper/2302.10329/) (2023) - [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021) - [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023) - [Explanations Can Reduce Overreliance on AI Systems During Decision-Making](https://mlsanity.com/paper/2212.06823/) (2022) - [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024) - [Can Large Language Models Reason and Plan?](https://mlsanity.com/paper/2403.04121/) (2024) - [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022) - [In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making](https://mlsanity.com/paper/2305.07722/) (2023) - [The Emerging Landscape of Explainable AI Planning and Decision Making](https://mlsanity.com/paper/2002.11697/) (2020) - [Explainable Planning](https://mlsanity.com/paper/1709.10256/) (2017) - [Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AI](https://mlsanity.com/paper/2105.05424/) (2021) ## Built on by (most cited first, 12 of 12) - [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025) - [Agentic AI Needs a Systems Theory](https://mlsanity.com/paper/2503.00237/) (2025) - [When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents](https://mlsanity.com/paper/2602.08235/) (2026) - [Completion neq Collaboration: Scaling Collaborative Effort with Agents](https://mlsanity.com/paper/2510.25744/) (2025) - [Modeling Distinct Human Interaction in Web Agents](https://mlsanity.com/paper/2602.17588/) (2026) - [MARK: Memory Augmented Refinement of Knowledge](https://mlsanity.com/paper/2505.05177/) (2025) - [A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models](https://mlsanity.com/paper/2504.02793/) (2025) - [VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluation](https://mlsanity.com/paper/2505.13577/) (2025) - [HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualization](https://mlsanity.com/paper/2606.26614/) (2026) - [Do LLMs Know What to Ask and When? Evaluating Multi-Turn Information Seeking](https://mlsanity.com/paper/2608.14808/) (2026) - [When Passing Tests Hides Vulnerabilities: An Empirical Study of Silent Failures in Agentic Systems](https://mlsanity.com/paper/2609.10548/) (2026) - [Using Natural Language for Human-Robot Collaboration in the Real World](https://mlsanity.com/paper/2508.11759/) (2025) ## Abstract (arXiv) Remarkable advancements in modern generative foundation models have enabled the development of sophisticated and highly capable autonomous agents that can observe their environment, invoke tools, and communicate with other agents to solve problems. Although such agents can communicate with users through natural language, their complexity and wide-ranging failure modes present novel challenges for human-AI interaction. Building on prior research and informed by a communication grounding perspective, we contribute to the study of human-agent communication by identifying and analyzing twelve key communication challenges that these systems pose. These include challenges in conveying information from the agent to the user, challenges in enabling the user to convey information to the agent, and overarching challenges that need to be considered across all human-agent communication. We illustrate each challenge through concrete examples and identify open directions of research. Our findings provide insights into critical gaps in human-agent communication research and serve as an urgent call for new design patterns, principles, and guidelines to support transparency and control in these systems.