Are Multimodal Large Language Models Pragmatically Competent Listeners in Simple Reference Resolution Tasks?
On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Abstract (arXiv)
We investigate the linguistic abilities of multimodal large language models in reference resolution tasks featuring simple yet abstract visual stimuli, such as color patches and color grids. Although the task may not seem challenging for today's language models, being straightforward for human dyads, we consider it to be a highly relevant probe of the pragmatic capabilities of MLLMs. Our results and analyses indeed suggest that basic pragmatic capabilities, such as context-dependent interpretation of color descriptions, still constitute major challenges for state-of-the-art MLLMs.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 28 references that are in the map. First posted on arXiv 13 Jun 2025.
Builds on
10 of 28 references.
Closest ancestors · 2
Same direction, newest first
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
- 2023Improved Baselines with Visual Instruction Tuning5 Oct
Other references · 8
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 119
- 2023Language Model Behavior: A Comprehensive Surveycited by 92
- 2017Colors in Context: A Pragmatic Neural Model for Grounded Language Understandingcited by 57
- 2024Language Models as Models of Languagecited by 24
- 2016Character Sequence Models for ColorfulWordscited by 5
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2025Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Modelscited by 11
- 2025On the Same Wavelength? Evaluating Pragmatic Reasoning in Language Models across Broad Conceptscited by 5
- 2026Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requestscited by 2
- 2026Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requests12 Jan 2026
- 2025Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models6 Oct 2025
- 2025On the Same Wavelength? Evaluating Pragmatic Reasoning in Language Models across Broad Concepts8 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025Evaluating Graphical Perception with Multimodal LLMs
- 2026Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
- 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
- 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models
- 2024A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment
- 2024Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach
- 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks
- 2025I Spy With My Model's Eye: Visual Search as a Behavioural Test for MLLMs