mlsanityData to 7 Oct 2026

Reference-Centric Models for Grounded Collaborative Dialogue

arXiv 2109.05042 ↗First posted 10 Sep 2021cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

We present a grounded neural dialogue model that successfully collaborates with people in a partially-observable reference game. We focus on a setting where two agents each observe an overlapping part of a world context and need to identify and agree on some object they share. Therefore, the agents should pool their information and communicate pragmatically to solve the task. Our dialogue agent accurately grounds referents from the partner's utterances using a structured reference resolver, conditions on these referents using a recurrent memory, and uses a pragmatic generation procedure to ensure the partner can resolve the references the agent produces. We evaluate on the OneCommon spatial grounding dialogue task (Udagawa and Aizawa 2019), involving a number of dots arranged on a board with continuously varying positions, sizes, and shades. Our agent substantially outperforms the previous state of the art for the task, obtaining a 20% relative improvement in successful task completion in self-play evaluations and a 50% relative improvement in success in human evaluations.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 29 of its 62 references that are in the map. First posted on arXiv 10 Sep 2021.

Builds on

29 of 62 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  4. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  5. 2016Visual Dialogcited by 377
  6. 2017A simple neural network module for relational reasoningcited by 345
  7. 2017Embodied Question Answeringcited by 335
  8. 2019Vision-and-Dialog Navigationcited by 246
  9. 2018Decoupling Strategy and Generation in Negotiation Dialoguescited by 156
  10. 2016Learning End-to-End Goal-Oriented Dialogcited by 104
  11. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  12. 2016A Joint Speaker-Listener-Reinforcer Model for Referring Expressionscited by 74
17 more
  1. 2016Reasoning About Pragmatics with Neural Listeners and Speakerscited by 68
  2. 2019Executing Instructions in Situated Collaborative Interactionscited by 68
  3. 2017Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddingscited by 59
  4. 2017Colors in Context: A Pragmatic Neural Model for Grounded Language Understandingcited by 57
  5. 2019The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialoguecited by 46
  6. 2018Talk the Walk: Navigating New York City through Grounded Dialoguecited by 44
  7. 2017Context-aware Captions from Context-agnostic Supervisioncited by 43
  8. 2018Pragmatically Informative Image Captioning with Character-Level Inferencecited by 42
  9. 2019A Natural Language Corpus of Common Grounding under Continuous and Partially-Observable Contextcited by 35
  10. 2017Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learningcited by 31
  11. 2020Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contextscited by 22
  12. 2017Naturalizing a Programming Language via Interactive Learningcited by 20
  13. 2019MeetUp! A Corpus of Joint Activity Dialogues in a Visual Environmentcited by 19
  14. 2020Learning to refer informatively by amortizing pragmatic reasoningcited by 17
  15. 2018Planning, Inference and Pragmatics in Sequential Language Gamescited by 12
  16. 2019An Annotated Corpus of Reference Resolution for Interpreting Common Groundingcited by 6
  17. 2020A Linguistic Analysis of Visually Grounded Dialogues Based on Spatial Expressionscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 8; 2024: 4; 2025: 0; 2026: 1.

15 papers.

Sort:

By area:

  1. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  2. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 129
  3. 2024Efficacy of Language Model Self-Play in Non-Zero-Sum Gamescited by 14
  4. 2022Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approachescited by 12
  5. 2024Conversational Grounding: Annotation and Analysis of Grounding Acts and Grounding Unitscited by 8
  6. 2023Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mindcited by 7
  7. 2023Asking More Informative Questions for Grounded Retrievalcited by 6
  8. 2023Symbolic Planning and Code Generation for Grounded Dialoguecited by 5
  9. 2023Pragmatic Inference with a CLIP Listener for Contrastive Captioningcited by 4
  10. 2024Amortizing Pragmatic Program Synthesis with Rankingscited by 3
  11. 2026Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Taskscited by 1
  12. 2022Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scenecited by 0
3 more
  1. 2023Core Challenges in Embodied Vision-Language Planningcited by 0
  2. 2023Listener Model for the PhotoBook Referential Game with CLIPScores as Implicit Reference Chaincited by 0
  3. 2023Amortizing Pragmatic Program Synthesis with Rankingscited by 0