# Coreference as an indicator of context scope in multimodal narrative

arXiv: https://arxiv.org/abs/2503.05298 · HTML: https://mlsanity.com/paper/2503.05298/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 7 Mar 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [MovieNet: A Holistic Dataset for Movie Understanding](https://mlsanity.com/paper/2007.10937/) (2020)
- [Visual Storytelling](https://mlsanity.com/paper/1604.03968/) (2016)
- [Content Planning for Neural Story Generation with Aristotelian Rescoring](https://mlsanity.com/paper/2009.09870/) (2020)
- [A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation](https://mlsanity.com/paper/1808.06945/) (2018)
- [People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text](https://mlsanity.com/paper/2501.15654/) (2025)
- [Coreference Resolution through a seq2seq Transition-Based System](https://mlsanity.com/paper/2211.12142/) (2022)
- [Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences](https://mlsanity.com/paper/2301.08571/) (2023)
- [Maverick: Efficient and Accurate Coreference Resolution Defying Recent Trends](https://mlsanity.com/paper/2407.21489/) (2024)
- [Grounding Language in Multi-Perspective Referential Communication](https://mlsanity.com/paper/2410.03959/) (2024)
- [Evaluating Document Coherence Modelling](https://mlsanity.com/paper/2103.10133/) (2021)
- [Detecting and Grounding Important Characters in Visual Stories](https://mlsanity.com/paper/2303.17647/) (2023)

## Abstract (arXiv)

We demonstrate that large multimodal language models differ substantially from humans in the distribution of coreferential expressions in a visual storytelling task. We introduce a number of metrics to quantify the characteristics of coreferential patterns in both human- and machine-written texts. Humans distribute coreferential expressions in a way that maintains consistency across texts and images, interleaving references to different entities in a highly varied way. Machines are less able to track mixed references, despite achieving perceived improvements in generation quality. Materials, metrics, and code for our study are available at https://github.com/GU-CLASP/coreference-context-scope.