mlsanityData to 7 Oct 2026

Exploring Plan Space through Conversation: An Agentic Framework for LLM-Mediated Explanations in Planning

arXiv 2603.02070 ↗First posted 2 Mar 2026cs.AI cs.CL cs.HCcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

When automating plan generation for a real-world sequential decision problem, the goal is often not to replace the human planner, but to facilitate an iterative reasoning and elicitation process, where the human's role is to guide the AI planner according to their preferences and expertise. In this context, explanations that respond to users' questions are crucial to improve their understanding of potential solutions and increase their trust in the system. To enable natural interaction with such a system, we present a multi-agent Large Language Model (LLM) architecture that is agnostic to the explanation framework and enables user- and context-dependent interactive explanations. We also describe an instantiation of this framework for goal-conflict explanations, which we use to conduct a user study comparing the LLM-powered interaction with a baseline template-based explanation interface.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 23 of its 48 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

23 of 48 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan
  2. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors21 Aug

Other references · 21

  1. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  2. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  3. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 251
  4. 2022TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversationscited by 65
  5. 2024On Limitations of the Transformer Architecturecited by 63
  6. 2023nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Modelscited by 57
  7. 2022Rethinking Explainability as a Dialogue: A Practitioner's Perspectivecited by 52
  8. 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
  9. 2025Emergent Coordination in Multi-Agent Language Modelscited by 22
  10. 2021Levels of explainable artificial intelligence for human-aligned conversational explanationscited by 21
  11. 2021Contrastive Explanations of Plans Through Model Restrictionscited by 16
  12. 2022From Black Boxes to Conversations: Incorporating XAI in a Conversational Agentcited by 14
9 more
  1. 2023InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanationscited by 11
  2. 2025OptiChat: Bridging Optimization Models and Practitioners with Large Language Modelscited by 8
  3. 2025Frontier Large Language Models Rival State-of-the-Art Plannerscited by 6
  4. 2023Efficiently Explaining CSPs with Unsatisfiable Subset Optimization (extended algorithms and examples)cited by 5
  5. 2023May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainabilitycited by 5
  6. 2023Reason to explain: Interactive contrastive explanations (REASONX)cited by 2
  7. 2024Modeling the Quality of Dialogical Explanationscited by 2
  8. 2025Investigating Co-Constructive Behavior of Large Language Models in Explanation Dialoguescited by 2
  9. 2024Trustworthy and Explainable Decision-Making for Workforce allocationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM Reasoner and Automated Planner: A new NPC approach
  2. 2024A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Models
  3. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)
  4. 2024Revealing the Barriers of Language Agents in Planning
  5. 2024TwoStep: Multi-agent Task Planning using Classical Planners and Large Language Models
  6. 2023Improving Planning with Large Language Models: A Modular Agentic Architecture
  7. 2026Training the Orchestrator: A Supervised Approach to End-to-End PDDL Planning with LLM Agents
  8. 2024Ask-before-Plan: Proactive Language Agents for Real-World Planning