mlsanityData to 7 Oct 2026

Reasoning About Pragmatics with Neural Listeners and Speakers

arXiv 1604.00562 ↗First posted 2 Apr 2016cs.CL cs.NEcited by 68

Not on the map. Why?

Abstract (arXiv)

We present a model for pragmatically describing scenes, in which contrastive behavior results from a combination of inference-driven pragmatics and learned semantics. Like previous learned approaches to language generation, our model uses a simple feature-driven architecture (here a pair of neural "listener" and "speaker" models) to ground language in the world. Like inference-driven approaches to pragmatics, our model actively reasons about listener behavior when selecting utterances. For training, our approach requires only ordinary captions, annotated _without_ demonstration of the pragmatic behavior the model ultimately exhibits. In human evaluations on a referring expression game, our approach succeeds 81% of the time, compared to a 69% success rate using existing techniques.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 11; 2023: 19; 2024: 13; 2025: 4; 2026: 9.

68 papers.

Sort:

By area:

  1. 2021Modeling Strong and Human-Like Gameplay with KL-Regularized Searchcited by 50
  2. 2021Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causescited by 43
  3. 2021From partners to populations: A hierarchical Bayesian account of coordination and conventioncited by 27
  4. 2021Interactive Learning from Activity Descriptioncited by 26
  5. 2022Image Retrieval from Contextual Descriptionscited by 26
  6. 2022How to talk so AI will learn: Instructions, descriptions, and autonomycited by 22
  7. 2023Human-like Few-Shot Learning via Bayesian Reasoning over Natural Languagecited by 21
  8. 2023Modeling Cross-Cultural Pragmatic Inference with Codenames Duetcited by 20
  9. 2022Affection: Learning Affective Explanations for Real-World Visual Datacited by 19
  10. 2024Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMscited by 18
  11. 2021Reference-Centric Models for Grounded Collaborative Dialoguecited by 15
  12. 2023Computational Language Acquisition with Theory of Mindcited by 15
38 more
  1. 2023Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalizationcited by 15
  2. 2022Learning to Mediate Disparities Towards Pragmatic Communicationcited by 14
  3. 2021The Emergence of the Shape Bias Results from Communicative Efficiencycited by 13
  4. 2021A Framework for Learning to Request Rich and Contextually Useful Information from Humanscited by 12
  5. 2022Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approachescited by 12
  6. 2023Towards More Human-like AI Communication: A Review of Emergent Communication Researchcited by 12
  7. 2023Object Recognition as Next Token Predictioncited by 12
  8. 2024RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Frameworkcited by 11
  9. 2023Language Models are Bounded Pragmatic Speakers: Understanding RLHF from a Bayesian Cognitive Modeling Perspectivecited by 10
  10. 2026Value of Information: A Framework for Human-Agent Communicationcited by 10
  11. 2021Calibrate your listeners! Robust communication-based training for pragmatic speakerscited by 9
  12. 2023Dealing with Semantic Underspecification in Multimodal NLPcited by 9
  13. 2022Emergence of hierarchical reference systems in multi-agent communicationcited by 8
  14. 2023DisCLIP: Open-Vocabulary Referring Expression Generationcited by 8
  15. 2022Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Modelscited by 7
  16. 2023Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mindcited by 7
  17. 2024Concept-Best-Matching: Evaluating Compositionality in Emergent Communicationcited by 7
  18. 2024CoGen: Learning from Feedback with Coupled Comprehension and Generationcited by 7
  19. 2024Are LLMs good pragmatic speakers?cited by 7
  20. 2021A practical introduction to the Rational Speech Act modeling frameworkcited by 6
  21. 2021Iterative Teacher-Aware Learningcited by 6
  22. 2023Evaluating Pragmatic Abilities of Image Captioners on A3DScited by 5
  23. 2023Generating Pragmatic Examples to Train Neural Program Synthesizerscited by 5
  24. 2022Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learningcited by 4
  25. 2023Pragmatic Inference with a CLIP Listener for Contrastive Captioningcited by 4
  26. 2023"We Demand Justice!": Towards Social Context Grounding of Political Textscited by 4
  27. 2022Emergence of Machine Language: Towards Symbolic Intelligence with Neural Networkscited by 3
  28. 2023Impressions: Understanding Visual Semiotics and Aesthetic Impactcited by 3
  29. 2023Which One? Leveraging Context Between Objects and Multiple Views for Language Groundingcited by 3
  30. 2023Modeling Boundedly Rational Agents with Latent Inference Budgetscited by 3
  31. 2024Representational Alignment Supports Effective Machine Teachingcited by 3
  32. 2024Amortizing Pragmatic Program Synthesis with Rankingscited by 3
  33. 2023PROGrasp: Pragmatic Human-Robot Communication for Object Graspingcited by 2
  34. 2024Cognitive Modeling with Scaffolded LLMs: A Case Study of Referential Expression Generationcited by 2
  35. 2024Communicating with Speakers and Listeners of Different Pragmatic Levelscited by 2
  36. 2024Self-correction is Not An Innate Capability in Language Modelscited by 2
  37. 2026When More Words Say Less: Decoupling Length and Specificity in Image Description Evaluationcited by 2
  38. 2021Scalable pragmatic communication via self-supervisioncited by 1

50 most cited of 68 All 68 (JSON)