mlsanityData to 14 Oct 2026

Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches

arXiv 2211.08371 ↗First posted 15 Nov 2022cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

People rely heavily on context to enrich meaning beyond what is literally said, enabling concise but effective communication. To interact successfully and naturally with people, user-facing artificial intelligence systems will require similar skills in pragmatics: relying on various types of context -- from shared linguistic goals and conventions, to the visual and embodied world -- to use language effectively. We survey existing grounded settings and pragmatic modeling approaches and analyze how the task goals, environmental contexts, and communicative affordances in each work enrich linguistic meaning. We present recommendations for future grounded task design to naturally elicit pragmatic phenomena, and suggest directions that focus on a broader range of communicative contexts and affordances.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 88 of its 232 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

88 of 232 references.

References · 88

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2020Learning to summarize from human feedbackcited by 2,588
  8. 2015VQA: Visual Question Answeringcited by 2,506
  9. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  10. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  11. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
38 more
  1. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  3. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  4. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  5. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  6. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
  7. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 459
  8. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  9. 2016Visual Dialogcited by 377
  10. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  11. 2016Cooperative Inverse Reinforcement Learningcited by 300
  12. 2019Vision-and-Dialog Navigationcited by 250
  13. 2017Inverse Reward Designcited by 249
  14. 2020Experience Grounds Languagecited by 242
  15. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  16. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
  17. 2019Connecting Vision and Language with Localized Narrativescited by 226
  18. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  19. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 209
  20. 2019Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Goodcited by 209
  21. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  22. 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 170
  23. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  24. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155
  25. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  26. 2021Collaborating with Humans without Human Datacited by 142
  27. 2022Linearly Mapping from Image to Text Spacecited by 142
  28. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  29. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 120
  30. 2016GuessWhat?! Visual object discovery through multi-modal dialoguecited by 113
  31. 2021Decontextualization: Making Sentences Stand-Alonecited by 102
  32. 2021MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuningcited by 85
  33. 2020Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESuppositioncited by 77
  34. 2019Executing Instructions in Situated Collaborative Interactionscited by 69
  35. 2016Reasoning About Pragmatics with Neural Listeners and Speakerscited by 68
  36. 2018Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 68
  37. 2017A Large Self-Annotated Corpus for Sarcasmcited by 64
  38. 2022The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMscited by 58

50 most cited of 88 All 88 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 2; 2026: 3.

12 papers.

  1. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  2. 2023Decision-Oriented Dialogue for Human-AI Collaborationcited by 29
  3. 2024Steering Language Models with Game-Theoretic Solverscited by 28
  4. 2024How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?cited by 27
  5. 2023Evaluating statistical language models as pragmatic reasonerscited by 15
  6. 2024Efficacy of Language Model Self-Play in Non-Zero-Sum Gamescited by 15
  7. 2025Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generationcited by 8
  8. 2023A framework for the emergence and analysis of language in social learning agentscited by 2
  9. 2025SITCOM: Scaling Inference-Time COMpute for VLAscited by 0
  10. 2026Pragmatics Meets Culture: Culturally-adapted Artwork Description Generation and Evaluationcited by 0
  11. 2026Beneath the Surface: Investigating LLMs' Capabilities for Communicating with Subtextcited by 0
  12. 2026A paradox of AI fluencycited by 0