mlsanityData to 14 Oct 2026

Coreference as an indicator of context scope in multimodal narrative

arXiv 2503.05298 ↗First posted 7 Mar 2025cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

We demonstrate that large multimodal language models differ substantially from humans in the distribution of coreferential expressions in a visual storytelling task. We introduce a number of metrics to quantify the characteristics of coreferential patterns in both human- and machine-written texts. Humans distribute coreferential expressions in a way that maintains consistency across texts and images, interleaving references to different entities in a highly varied way. Machines are less able to track mixed references, despite achieving perceived improvements in generation quality. Materials, metrics, and code for our study are available at https://github.com/GU-CLASP/coreference-context-scope.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 41 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

18 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding13 Dec
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep

Other references · 15

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  3. 2018Hierarchical Neural Story Generationcited by 1,193
  4. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  5. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  6. 2016Visual Storytellingcited by 176
  7. 2020Content Planning for Neural Story Generation with Aristotelian Rescoringcited by 92
  8. 2018A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generationcited by 42
  9. 2025People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated textcited by 40
  10. 2022Coreference Resolution through a seq2seq Transition-Based Systemcited by 34
  11. 2023Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequencescited by 31
  12. 2024Maverick: Efficient and Accurate Coreference Resolution Defying Recent Trendscited by 19
3 more
  1. 2024Grounding Language in Multi-Perspective Referential Communicationcited by 12
  2. 2021Evaluating Document Coherence Modellingcited by 8
  3. 2023Detecting and Grounding Important Characters in Visual Storiescited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Humans vs Vision-Language Models: A Unified Measure of Narrative Coherence
  2. 2025I Prompt, it Generates, we Negotiate. Exploring Text-Image Intertextuality in Human-AI Co-Creation of Visual Narratives with VLMs ↗
  3. 2024Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you!
  4. 2026Narrative Flattening: How Post-Training Compresses Thematic, Affective, and Stylistic Variation in LLM Fiction
  5. 2026What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation
  6. 2026COHERENCE: Benchmarking Fine-Grained Image-Text Alignment in Interleaved Multimodal Contexts
  7. 2024Are Large Language Models Capable of Generating Human-Level Narratives?
  8. 2024Improving Visual Storytelling with Multimodal Large Language Models