mlsanityData to 7 Oct 2026

Ask-before-Plan: Proactive Language Agents for Real-World Planning

arXiv 2406.12639 ↗First posted 18 Jun 2024cs.CL cs.AIcited by 59

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The evolution of large language models (LLMs) has enhanced the planning capabilities of language agents in diverse real-world scenarios. Despite these advancements, the potential of LLM-powered agents to comprehend ambiguous user instructions for reasoning and decision-making is still under exploration. In this work, we introduce a new task, Proactive Agent Planning, which requires language agents to predict clarification needs based on user-agent conversation and agent-environment interaction, invoke external tools to collect valid information, and generate a plan to fulfill the user's demands. To study this practical problem, we establish a new benchmark dataset, Ask-before-Plan. To tackle the deficiency of LLMs in proactive planning, we propose a novel multi-agent framework, Clarification-Execution-Planning (textttCEP), which consists of three agents specialized in clarification, execution, and planning. We introduce the trajectory tuning scheme for the clarification agent and static execution agent, as well as the memory recollection mechanism for the dynamic execution agent. Extensive evaluations and comprehensive analyses conducted on the Ask-before-Plan dataset validate the effectiveness of our proposed framework.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 26 of its 44 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

26 of 44 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Adaptive In-conversation Team Building for Language Model Agents29 May
  2. 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents14 Feb
  3. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agents2 Feb
  4. 2024Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents1 Feb
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  6. 2023A Survey on Large Language Model based Autonomous Agents22 Aug

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  6. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  7. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  8. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 215
  9. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 210
  10. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 115
  11. 2018Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Informationcited by 110
  12. 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 99
8 more
  1. 2022CLAM: Selective Clarification for Ambiguous Questions with Generative Language Modelscited by 66
  2. 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
  3. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  4. 2022PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Financecited by 45
  5. 2024On the Multi-turn Instruction Following for Conversational Web Agentscited by 42
  6. 2022ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situationscited by 36
  7. 2022INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactionscited by 27
  8. 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agentscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 25; 2026: 30.

59 papers.

Sort:

By area:

  1. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 56
  2. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  3. 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 44
  4. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  5. 2025Structured Uncertainty guided Clarification for LLM Agentscited by 34
  6. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 31
  7. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 28
  8. 2025PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agentscited by 15
  9. 2025ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agentcited by 14
  10. 2024Depression Diagnosis Dialogue Simulation: Self-improving Psychiatrist with Tertiary Memorycited by 13
  11. 2025TP-RAG: Benchmarking Retrieval-Augmented Large Language Model Agents for Spatiotemporal-Aware Travel Planningcited by 12
  12. 2025Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimizationcited by 10
38 more
  1. 2026Value of Information: A Framework for Human-Agent Communicationcited by 10
  2. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 10
  3. 2025RETAIL: Towards Real-world Travel Planning for Large Language Modelscited by 9
  4. 2025ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agentscited by 9
  5. 2025Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agentscited by 9
  6. 2026Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learningcited by 9
  7. 2025LLAMAPIE: Proactive In-Ear Conversation Assistantscited by 8
  8. 2024Towards Full Delegation: Designing Ideal Agentic Behaviors for Travel Planningcited by 7
  9. 2025ProAgent: Harnessing On-Demand Sensory Contexts for Proactive LLM Agent Systems in the Wildcited by 7
  10. 2026Multi-User Large Language Model Agentscited by 6
  11. 2026Uncertainty-Aware Clarification in LLM Agents with Information Gaincited by 6
  12. 2026AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraintscited by 6
  13. 2025AppAgent-Pro: A Proactive GUI Agent System for Multidomain Information Integration and User Assistancecited by 5
  14. 2025CaRT: Teaching LLM Agents to Know When They Know Enoughcited by 5
  15. 2026ProactiveMobile: A Comprehensive Benchmark for Boosting Proactive Intelligence on Mobile Devicescited by 5
  16. 2026GroupTravelBench: Benchmarking LLM Agents on Multi-Person Travel Planningcited by 5
  17. 2024Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Taskscited by 4
  18. 2025Zero-Shot and Efficient Clarification Need Prediction in Conversational Searchcited by 4
  19. 2024AssistantX: An LLM-Powered Proactive Assistant in Collaborative Human-Populated Environmentcited by 3
  20. 2026IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agentscited by 3
  21. 2026SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Realitycited by 3
  22. 2026PRISM: Festina Lente Proactivity -- Risk-Sensitive, Uncertainty-Aware Deliberation for Proactive Agentscited by 3
  23. 2026Egocentric Co-Pilot: Web-Native Smart-Glasses Agents for Assistive Egocentric AIcited by 3
  24. 2025Multilingual Agent-Based World Modeling for Social Sciencecited by 2
  25. 2025LLM-Driven Preference Data Synthesis for Proactive Prediction of the Next User Utterance in Human-Machine Dialoguecited by 2
  26. 2026PROPER Agents: Proactivity Driven Personalized Agents for Advancing Knowledge Gap Navigationcited by 2
  27. 2026Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?cited by 2
  28. 2026Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agentscited by 2
  29. 2026Ask Now, Use Later: Benchmarking the Proactivity Gap in Long-Lived LLM Agentscited by 2
  30. 2026Do Proactive Agents Really Need an LLM to Decide When to Wake and What to Anchor?cited by 2
  31. 2025A Data Synthesis Method Driven by Large Language Models for Proactive Mining of Implicit User Intentions in Tourismcited by 1
  32. 2025ReInAgent: A Context-Aware GUI Agent Enabling Human-in-the-Loop Mobile Task Navigationcited by 1
  33. 2025How2: How to learn from procedural How-to questionscited by 1
  34. 2026AWARE-US: Preference-Aware Infeasibility Resolution in Tool-Calling Agentscited by 1
  35. 2026Game of Thought: Robust Information Seeking with Large Language Models Using Game Theorycited by 1
  36. 2026Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvementcited by 1
  37. 2026"Excuse me, may I say something..." CoLabScience, A Proactive AI Assistant for Biomedical Discovery and LLM-Expert Collaborationscited by 1
  38. 2026Clarify Before Executing: A Self-Evolving Agent for Resolving Intent Asymmetry in 3D Tool Orchestrationcited by 1

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks
  2. 2024Revealing the Barriers of Language Agents in Planning
  3. 2024One STEP at a time: Language Agents are Stepwise Planners
  4. 2024Query-Efficient Planning with Language Models
  5. 2026Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning
  6. 2024Understanding the planning of LLM agents: A survey
  7. 2025Pre-Act: Multi-Step Planning and Reasoning Improves Acting in LLM Agents
  8. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation