mlsanityData to 7 Oct 2026

Grounding 'Grounding' in NLP

arXiv 2106.02192 ↗First posted 4 Jun 2021cs.CLcited by 44

Not on the map. Why?

Abstract (arXiv)

The NLP community has seen substantial recent interest in grounding to facilitate interaction between language technologies and the world. However, as a community, we use the term broadly to reference any linking of text to data or non-textual modality. In contrast, Cognitive Science more formally defines "grounding" as the process of establishing what mutual information is required for successful communication between two interlocutors -- a definition which might implicitly capture the NLP usage but differs in intent and scope. We investigate the gap between these definitions and seek answers to the following questions: (1) What aspects of grounding are missing from NLP tasks? Here we present the dimensions of coordination, purviews and constraints. (2) How is the term "grounding" used in the current research? We study the trends in datasets, domains, and tasks introduced in recent NLP conferences. And finally, (3) How to advance our current definition to bridge the gap with Cognitive Science? We present ways to both create new tasks or repurpose existing ones to make advancements towards achieving a more complete sense of grounding.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 67 of its 169 references that are in the map. First posted on arXiv 4 Jun 2021.

Builds on

67 of 169 references.

References · 67

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2020Learning to summarize from human feedbackcited by 2,562
  3. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  5. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
  6. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  7. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  8. 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
  9. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  10. 2020Experience Grounds Languagecited by 241
  11. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  12. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
38 more
  1. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 153
  2. 2016GuessWhat?! Visual object discovery through multi-modal dialoguecited by 112
  3. 2018Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Informationcited by 110
  4. 2018A Dataset for Document Grounded Conversationscited by 104
  5. 2019Learning from Dialogue after Deployment: Feed Yourself, Chatbot!cited by 102
  6. 2018Image Chat: Engaging Grounded Conversationscited by 76
  7. 2020Grounded Adaptation for Zero-shot Executable Semantic Parsingcited by 76
  8. 2019Weakly-Supervised Spatio-Temporally Grounding Natural Sentence in Videocited by 60
  9. 2019Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphscited by 59
  10. 2017Colors in Context: A Pragmatic Neural Model for Grounded Language Understandingcited by 57
  11. 2020ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)cited by 55
  12. 2019Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigationcited by 54
  13. 2019Visually Grounded Neural Syntax Acquisitioncited by 48
  14. 2017STAIR Captions: Constructing a Large-Scale Japanese Image Caption Datasetcited by 47
  15. 2019The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialoguecited by 46
  16. 2017Representations of language in a model of visually grounded speech signalcited by 37
  17. 2015Text to 3D Scene Generation with Rich Lexical Groundingcited by 36
  18. 2019Incremental Transformer with Deliberation Decoder for Document Grounded Conversationscited by 35
  19. 2020Visually Grounded Compound PCFGscited by 34
  20. 2019Large-scale representation learning from visually grounded untranscribed speechcited by 33
  21. 2020Explaining Question Answering Models through Text Generationcited by 33
  22. 2020Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressionscited by 33
  23. 2020RMM: A Recursive Mental Model for Dialog Navigationcited by 32
  24. 2020ClarQ: A large-scale and diverse dataset for Clarification Question Generationcited by 31
  25. 2017Encoding of phonology in a recurrent neural model of grounded speechcited by 27
  26. 2017Representation Learning for Grounded Spatial Reasoningcited by 23
  27. 2018A Visual Attention Grounding Neural Model for Multimodal Machine Translationcited by 22
  28. 2020Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contextscited by 22
  29. 2020Grounding Conversations with Improvised Dialoguescited by 21
  30. 2019Zero-Shot Open Entity Typing as Type-Compatible Groundingcited by 19
  31. 2020Visual Grounding Methods for VQA are Working for the Wrong Reasons!cited by 18
  32. 2019"My Way of Telling a Story": Persona based Grounded Story Generationcited by 17
  33. 2019Word Recognition, Competition, and Activation in a Model of Visually Grounded Speechcited by 13
  34. 2021A recipe for annotating grounded clarificationscited by 13
  35. 2018Grounded Textual Entailmentcited by 10
  36. 2018Symbolic inductive bias for visually grounded learning of spoken languagecited by 10
  37. 2016Equation Parsing: Mapping Sentences to Grounded Equationscited by 9
  38. 2019Towards Content Transfer through Grounded Text Generationcited by 9

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 12; 2023: 13; 2024: 10; 2025: 5; 2026: 3.

44 papers.

Sort:

By area:

  1. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
  2. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  3. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Groundingcited by 78
  4. 2022Enabling Conversational Interaction with Mobile UI using Large Language Modelscited by 66
  5. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  6. 2024TopViewRS: Vision-Language Models as Top-View Spatial Reasonerscited by 48
  7. 2023Grounding Gaps in Language Model Generationscited by 45
  8. 2023The Vector Grounding Problemcited by 39
  9. 2024Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environmentscited by 39
  10. 20243D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucinationcited by 36
  11. 2024Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questionscited by 32
  12. 2023Are Large Language Models Temporally Grounded?cited by 25
32 more
  1. 2021Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanationscited by 24
  2. 2022Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inferencecited by 17
  3. 2023Evaluating Generative Ad Hoc Information Retrievalcited by 11
  4. 2022MUG: Interactive Multimodal Grounding on User Interfacescited by 9
  5. 2025Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questionscited by 9
  6. 2023What A Situated Language-Using Agent Must be Able to Do: A Top-Down Analysiscited by 8
  7. 2023GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regionscited by 8
  8. 2024Grounding from an AI and Cognitive Science Lenscited by 6
  9. 2025Enhancing Interpretable Image Classification Through LLM Agents and Conditional Concept Bottleneck Modelscited by 6
  10. 2022Do BERTs Learn to Use Browser User Interface? Exploring Multi-Step Tasks with Unified Vision-and-Language BERTscited by 5
  11. 2023Learning to Summarize and Answer Questions about a Virtual Robot's Past Actionscited by 5
  12. 2023Minimum Levels of Interpretability for Artificial Moral Agentscited by 5
  13. 2022Summarizing a virtual robot's past actions in natural languagecited by 3
  14. 2022Multilingual Event Linking to Wikidatacited by 3
  15. 2022Language Models Understand Us, Poorlycited by 3
  16. 2024Situated Instruction Followingcited by 3
  17. 2025Enhancing Code Generation via Bidirectional Comment-Level Mutual Groundingcited by 3
  18. 2022Extending Phrase Grounding with Pronouns in Visual Dialoguescited by 2
  19. 2024Tailoring Vaccine Messaging with Common-Ground Opinionscited by 2
  20. 2024LLM-Human Pipeline for Cultural Context Grounding of Conversationscited by 2
  21. 2022Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunitiescited by 1
  22. 2022Grounding Scene Graphs on Natural Images via Visio-Lingual Message Passingcited by 1
  23. 2023On Degrees of Freedom in Defining and Testing Natural Language Understandingcited by 1
  24. 2023Grounded and Well-rounded: A Methodological Approach to the Study of Cross-modal and Cross-lingual Groundingcited by 1
  25. 2024Measuring the Groundedness of Legal Question-Answering Systemscited by 1
  26. 2026Talk is Cheap, Communication is Hard: Dynamic Grounding Failures and Repair in Multi-Agent Negotiationcited by 1
  27. 2023Hierarchical Event Groundingcited by 0
  28. 2023Describe me an Aucklet: Generating Grounded Perceptual Category Descriptionscited by 0
  29. 2023On General Language Understandingcited by 0
  30. 2025Federated Learning Inspired Fuzzy Systems: Decentralized Rule Updating for Privacy and Scalable Decision Makingcited by 0
  31. 2026Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision-Language Modelscited by 0
  32. 2026LLMs Infer Cultural Context but Fail to Apply It When Respondingcited by 0