mlsanityData to 7 Oct 2026

Unified Pragmatic Models for Generating and Following Instructions

arXiv 1711.04987 ↗First posted 14 Nov 2017cs.CLcited by 55

Not on the map. Why?

Abstract (arXiv)

We show that explicit pragmatic inference aids in correctly generating and following natural language instructions for complex, sequential tasks. Our pragmatics-enabled models reason about why speakers produce certain instructions, and about how listeners will react upon hearing them. Like previous pragmatic models, we use learned base listener and speaker models to build a pragmatic speaker that uses the base listener to simulate the interpretation of candidate descriptions, and a pragmatic listener that reasons counterfactually about alternative descriptions. We extend these models to tasks with sequential structure. Evaluation of language generation and interpretation shows that pragmatic inference improves state-of-the-art listener models (at correctly interpreting human instructions) and speaker models (at producing instructions correctly interpreted by humans) in diverse settings.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 13; 2023: 13; 2024: 15; 2025: 3; 2026: 2.

55 papers.

Sort:

By area:

  1. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  2. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 129
  3. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  4. 2022Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigationcited by 50
  5. 2022Inferring Rewards from Language in Contextcited by 50
  6. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  7. 2023Lana: A Language-Capable Navigator for Instruction Following and Generationcited by 39
  8. 2021Few-shot Language Coordination by Modeling Theory of Mindcited by 32
  9. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  10. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  11. 2021Interactive Learning from Activity Descriptioncited by 26
  12. 2022Few-shot Subgoal Planning with Language Modelscited by 22
38 more
  1. 2022FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigationcited by 22
  2. 2022How to talk so AI will learn: Instructions, descriptions, and autonomycited by 22
  3. 2023Modeling Cross-Cultural Pragmatic Inference with Codenames Duetcited by 20
  4. 2024Navigation Instruction Generation with BEV Perception and Large Language Modelscited by 20
  5. 2021Continual Learning for Grounded Instruction Generation by Observing Human Following Behaviorcited by 19
  6. 2023HandMeThat: Human-Robot Communication in Physical and Social Environmentscited by 19
  7. 2024Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMscited by 18
  8. 2025Morae: Proactively Pausing UI Agents for User Choicescited by 18
  9. 2024Policy Improvement using Language Feedback Modelscited by 16
  10. 2021Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsingcited by 15
  11. 2022Learning to Mediate Disparities Towards Pragmatic Communicationcited by 14
  12. 2024LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Modelscited by 14
  13. 2021A Framework for Learning to Request Rich and Contextually Useful Information from Humanscited by 12
  14. 2022Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approachescited by 12
  15. 2024Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoningcited by 11
  16. 2021SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmarkcited by 10
  17. 2022Innovations in Neural Data-to-text Generation: A Surveycited by 10
  18. 2023Language Models are Bounded Pragmatic Speakers: Understanding RLHF from a Bayesian Cognitive Modeling Perspectivecited by 10
  19. 2022Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Modelscited by 7
  20. 2023Hallucination Detection for Grounded Instruction Generationcited by 7
  21. 2024CoGen: Learning from Feedback with Coupled Comprehension and Generationcited by 7
  22. 2022"Mama Always Had a Way of Explaining Things So I Could Understand'': A Dialogue Corpus for Learning to Construct Explanationscited by 6
  23. 2022LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-trainingcited by 5
  24. 2023When and What to Ask Through World States and Text Instructions: IGLU NLP Challenge Solutioncited by 5
  25. 2023Generating Pragmatic Examples to Train Neural Program Synthesizerscited by 5
  26. 2023Masked Path Modeling for Vision-and-Language Navigationcited by 4
  27. 2023Pragmatic Inference with a CLIP Listener for Contrastive Captioningcited by 4
  28. 2022Know your audience: specializing grounded language models with listener subtractioncited by 3
  29. 2024Amortizing Pragmatic Program Synthesis with Rankingscited by 3
  30. 2024Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Correctionscited by 2
  31. 2024Modeling the Quality of Dialogical Explanationscited by 2
  32. 2024Communicating with Speakers and Listeners of Different Pragmatic Levelscited by 2
  33. 2024Self-correction is Not An Innate Capability in Language Modelscited by 2
  34. 2021When in Doubt: Improving Classification Performance with Alternating Normalizationcited by 1
  35. 2023CB2: Collaborative Natural Language Interaction Research Platformcited by 1
  36. 2026Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Taskscited by 1
  37. 2022Color Overmodification Emerges from Data-Driven Learning and Pragmatic Reasoningcited by 0
  38. 2023Core Challenges in Embodied Vision-Language Planningcited by 0

50 most cited of 55 All 55 (JSON)