mlsanityData to 14 Oct 2026

Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding

arXiv 1703.10186 ↗First posted 29 Mar 2017cs.CLcited by 57

Not on the map. Why?

Abstract (arXiv)

We present a model of pragmatic referring expression interpretation in a grounded communication task (identifying colors from descriptions) that draws upon predictions from two recurrent neural network classifiers, a speaker and a listener, unified by a recursive pragmatic reasoning framework. Experiments show that this combined pragmatic model interprets color descriptions more accurately than the classifiers from which it is built, and that much of this improvement results from combining the speaker and listener perspectives. We observe that pragmatic reasoning helps primarily in the hardest cases: when the model must distinguish very similar colors, or when few utterances adequately express the target color. Our findings make use of a newly-collected corpus of human utterances in color reference games, which exhibit a variety of pragmatic behaviors. We also show that the embedded speaker model reproduces many of these pragmatic behaviors.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 8; 2023: 14; 2024: 5; 2025: 11; 2026: 7.

57 papers.

Sort:

By area:

  1. 2021ArtEmis: Affective Language for Visual Artcited by 104
  2. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 60
  3. 2022Inferring Rewards from Language in Contextcited by 50
  4. 2021Grounding 'Grounding' in NLPcited by 44
  5. 2025Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challengescited by 31
  6. 2021From partners to populations: A hierarchical Bayesian account of coordination and conventioncited by 27
  7. 2022How to talk so AI will learn: Instructions, descriptions, and autonomycited by 22
  8. 2023Modeling Cross-Cultural Pragmatic Inference with Codenames Duetcited by 20
  9. 2022TriCoLo: Trimodal Contrastive Loss for Text to Shape Retrievalcited by 18
  10. 2024Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMscited by 18
  11. 2021Reference-Centric Models for Grounded Collaborative Dialoguecited by 15
  12. 2023Computational Language Acquisition with Theory of Mindcited by 15
38 more
  1. 2022Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approachescited by 12
  2. 2026Value of Information: A Framework for Human-Agent Communicationcited by 12
  3. 2022Innovations in Neural Data-to-text Generation: A Surveycited by 10
  4. 2021Maintaining Common Ground in Dynamic Environmentscited by 9
  5. 2021Analysis of Language Change in Collaborative Instruction Followingcited by 9
  6. 2021Calibrate your listeners! Robust communication-based training for pragmatic speakerscited by 9
  7. 2023Dealing with Semantic Underspecification in Multimodal NLPcited by 9
  8. 2025Measuring How (Not Just Whether) VLMs Build Common Groundcited by 8
  9. 2023Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mindcited by 7
  10. 2024CoGen: Learning from Feedback with Coupled Comprehension and Generationcited by 7
  11. 2024Are LLMs good pragmatic speakers?cited by 7
  12. 2021A practical introduction to the Rational Speech Act modeling frameworkcited by 6
  13. 2021Shades of confusion: Lexical uncertainty modulates ad hoc coordination in an interactive communication taskcited by 5
  14. 2023Multimodal Color Recommendation in Vector Graphic Documentscited by 5
  15. 2022Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learningcited by 4
  16. 2023Pragmatic Inference with a CLIP Listener for Contrastive Captioningcited by 4
  17. 2023Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoningcited by 4
  18. 2023Perceptual Structure in the Absence of Grounding for LLMs: The Impact of Abstractedness and Subjectivity in Color Languagecited by 4
  19. 2021Generating Compositional Color Representations from Textcited by 3
  20. 2022Resolving Indirect Referring Expressions for Entity Selectioncited by 3
  21. 2023Which One? Leveraging Context Between Objects and Multiple Views for Language Groundingcited by 3
  22. 2023Modeling Boundedly Rational Agents with Latent Inference Budgetscited by 3
  23. 2024Amortizing Pragmatic Program Synthesis with Rankingscited by 3
  24. 2024Why do objects have many names? A study on word informativeness in language use and lexical systemscited by 3
  25. 2025Are Multimodal Large Language Models Pragmatically Competent Listeners in Simple Reference Resolution Tasks?cited by 3
  26. 2023PROGrasp: Pragmatic Human-Robot Communication for Object Graspingcited by 2
  27. 2025Color Bind: Exploring Color Perception in Text-to-Image Modelscited by 2
  28. 2025NeLLCom-Lex: A Neural-agent Framework to Study the Interplay between Lexical Systems and Language Usecited by 2
  29. 2026Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requestscited by 2
  30. 2021Scalable pragmatic communication via self-supervisioncited by 1
  31. 2021When in Doubt: Improving Classification Performance with Alternating Normalizationcited by 1
  32. 2023Investigating Reinforcement Learning for Communication Strategies in a Task-Initiative Settingcited by 1
  33. 2025Rational Retrieval Acts: Leveraging Pragmatic Reasoning to Improve Sparse Retrievalcited by 1
  34. 2025Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generationcited by 1
  35. 2026LinguaGame: A Linguistically Grounded Game-Theoretic Paradigm for Multi-Agent Dialogue Generationcited by 1
  36. 2026Moral Mazes in the Era of LLMscited by 1
  37. 2022Color Overmodification Emerges from Data-Driven Learning and Pragmatic Reasoningcited by 0
  38. 2023Pragmatic Reasoning in Structured Signaling Gamescited by 0

50 most cited of 57 All 57 (JSON)