mlsanityData to 7 Oct 2026

Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners

arXiv 2307.01928 ↗First posted 4 Jul 2023cs.RO cs.AI stat.APcited by 298

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning -- that may provide utility for robots, but remain prone to confidently hallucinated predictions. In this work, we present KnowNo, which is a framework for measuring and aligning the uncertainty of LLM-based planners such that they know when they don't know and ask for help when needed. KnowNo builds on the theory of conformal prediction to provide statistical guarantees on task completion while minimizing human help in complex multi-step planning settings. Experiments across a variety of simulated and real robot setups that involve tasks with different modes of ambiguity (e.g., from spatial to numeric uncertainties, from human preferences to Winograd schemas) show that KnowNo performs favorably over modern baselines (which may involve ensembles or extensive prompt tuning) in terms of improving efficiency and autonomy, while providing formal assurances. KnowNo can be used with LLMs out of the box without model-finetuning, and suggests a promising lightweight approach to modeling uncertainty that can complement and scale with the growing capabilities of foundation models. Website: https://robot-help.github.io

Key facts. Cited by 298 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 54 of its 67 references that are in the map. First posted on arXiv 4 Jul 2023.

Builds on

54 of 67 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Task and Motion Planning with Large Language Models for Object Rearrangement10 Mar
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Models12 Jul

Other references · 52

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  7. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  9. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  10. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  11. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  12. 2023PaLM 2 Technical Reportcited by 1,154
38 more
  1. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  2. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  3. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  4. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  6. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  7. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  8. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  9. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  10. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  11. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  12. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 303
  13. 2022Confident Adaptive Language Modelingcited by 297
  14. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  15. 2019Vision-and-Dialog Navigationcited by 246
  16. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  17. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  18. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  19. 2023Conformal Language Modelingcited by 163
  20. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 132
  21. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 131
  22. 2018Correcting Length Bias in Neural Machine Translationcited by 119
  23. 2022Safe Planning in Dynamic Environments using Conformal Predictioncited by 112
  24. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  25. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  26. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  27. 2018Quantifying Uncertainties in Natural Language Processing Taskscited by 79
  28. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  29. 2022DialFRED: Dialogue-Enabled Agents for Embodied Instruction Followingcited by 63
  30. 2022Adaptive Conformal Prediction for Motion Planning among Dynamic Agentscited by 46
  31. 2021Sample-Efficient Safety Assurances using Conformal Predictioncited by 41
  32. 2022Conformal Prediction for STL Runtime Verificationcited by 34
  33. 2023CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agentscited by 28
  34. 2020The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigationcited by 26
  35. 2022Leveraging Language for Accelerated Learning of Tool Manipulationcited by 26
  36. 2022Efficient and Differentiable Conformal Prediction with General Function Classescited by 25
  37. 2021Conformal prediction for text infilling and part-of-speech predictioncited by 22
  38. 2023Active Reward Learning from Online Preferencescited by 12

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 30; 2024: 93; 2025: 81; 2026: 94.

298 papers.

Sort:

By area:

  1. 2023Cognitive Architectures for Language Agentscited by 422
  2. 2023FireAct: Toward Language Agent Fine-tuningcited by 247
  3. 2023Universal Self-Consistency for Large Language Model Generationcited by 198
  4. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  5. 2024Language Models with Conformal Factuality Guaranteescited by 128
  6. 2023AutoMix: Automatically Mixing Language Modelscited by 116
  7. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  8. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 113
  9. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  10. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  11. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  12. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
38 more
  1. 2023Self-Evaluation Improves Selective Generation in Large Language Modelscited by 74
  2. 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
  3. 2024Safeguarding Large Language Models: A Surveycited by 73
  4. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 63
  5. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  6. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  7. 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 56
  8. 2023Conformal Prediction for Deep Classifier via Label Rankingcited by 52
  9. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  10. 2024Conformal Alignment: Knowing When to Trust Foundation Models with Guaranteescited by 50
  11. 2025Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policiescited by 49
  12. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  13. 2023Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictionscited by 46
  14. 2024AGILE: A Novel Reinforcement Learning Framework of LLM Agentscited by 45
  15. 2023Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Correctionscited by 44
  16. 2024Multicalibration for Confidence Scoring in LLMscited by 44
  17. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  18. 2023Creative Robot Tool Use with Large Language Modelscited by 42
  19. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  20. 2026Code as Agent Harnesscited by 40
  21. 2023Interactive Task Planning with Language Modelscited by 36
  22. 2024VLM See, Robot Do: Human Demo Video to Robot Action Plan via Vision Language Modelcited by 36
  23. 2024Conformal Prediction Sets Improve Human Decision Makingcited by 35
  24. 2023Grasp-Anything: Large-scale Grasp Dataset from Foundation Modelscited by 33
  25. 2024Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languagescited by 32
  26. 2024VLMPC: Vision-Language Model Predictive Control for Robotic Manipulationcited by 32
  27. 2023Think, Act, and Ask: Open-World Interactive Personalized Robot Navigationcited by 31
  28. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
  29. 20253D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Taskscited by 30
  30. 2024Towards Uncertainty-Aware Language Agentcited by 29
  31. 2023Interactive Robot Learning from Verbal Correctioncited by 28
  32. 2024A Survey of Useful LLM Evaluationcited by 28
  33. 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 28
  34. 2023Interactive Planning Using Large Language Models for Partially Observable Robotics Taskscited by 27
  35. 2024DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiencescited by 27
  36. 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agentscited by 27
  37. 2024Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguitycited by 26
  38. 2023Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferencescited by 25

50 most cited of 298 All 298 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Can LLM-Reasoning Models Replace Classical Planning? A Benchmark Study
  2. 2026GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning
  3. 2025Using Language Models as Closed-Loop High-Level Planners for Robotics Applications: A Brief Overview and Benchmarks
  4. 2023DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment
  5. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
  6. 2026STEP: State-Aware Task Estimation and Planning with Multi-Modal LLMs for Human-Robot Collaboration
  7. 2025Constrained Natural Language Action Planning for Resilient Embodied Systems
  8. 2024Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Models