mlsanityData to 14 Oct 2026

Who are you referring to? Coreference resolution in image narrations

arXiv 2211.14563 ↗First posted 26 Nov 2022cs.CV cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Coreference resolution aims to identify words and phrases which refer to same entity in a text, a core task in natural language processing. In this paper, we extend this task to resolving coreferences in long-form narrations of visual scenes. First we introduce a new dataset with annotated coreference chains and their bounding boxes, as most existing image-text datasets only contain short sentences without coreferring expressions or labeled chains. We propose a new technique that learns to identify coreference chains using weak supervision, only from image-text pairs and a regularization using prior linguistic knowledge. Our model yields large performance gains over several strong baselines in resolving coreferences. We also show that coreference resolution helps improving grounding narratives in images.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 41 of its 71 references that are in the map. First posted on arXiv 26 Nov 2022.

Builds on

41 of 71 references.

References · 41

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  5. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  6. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  7. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  8. 2021Grounded Language-Image Pre-trainingcited by 1,156
  9. 2016Modeling Context in Referring Expressionscited by 1,063
  10. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  11. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
  12. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
29 more
  1. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  2. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  3. 2016Visual Dialogcited by 377
  4. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
  5. 2017End-to-end Neural Coreference Resolutioncited by 241
  6. 2019Connecting Vision and Language with Localized Narrativescited by 226
  7. 2018Higher-order Coreference Resolution with Coarse-to-fine Inferencecited by 150
  8. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  9. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 140
  10. 2020Contrastive Learning for Weakly Supervised Phrase Groundingcited by 102
  11. 2018Visual Coreference Resolution in Visual Dialog using Neural Module Networkscited by 54
  12. 2019Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignmentcited by 51
  13. 2020Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillationcited by 51
  14. 2016Improving Coreference Resolution by Learning Entity-Level Distributed Representationscited by 50
  15. 2018Rethinking Diversified and Discriminative Proposal Generation for Visual Groundingcited by 49
  16. 2018Neural Abstractive Text Summarization with Sequence-to-Sequence Modelscited by 39
  17. 2019Adaptive Reconstruction Network for Weakly Supervised Referring Expression Groundingcited by 37
  18. 2021Relation-aware Instance Refinement for Weakly Supervised Visual Groundingcited by 35
  19. 2017Query-guided Regression Network with Context Policy for Phrase Groundingcited by 34
  20. 2018Anaphora and Coreference Resolution: A Reviewcited by 31
  21. 2018PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolutioncited by 30
  22. 2022Entity-enhanced Adaptive Reconstruction Network for Weakly Supervised Referring Expression Groundingcited by 22
  23. 2019Knowledge-guided Pairwise Reconstruction Network for Weakly Supervised Referring Expression Groundingcited by 21
  24. 2016Learning Global Features for Coreference Resolutioncited by 20
  25. 2021Who's Waldo? Linking People Across Text and Imagescited by 16
  26. 2017Generating Descriptions with Grounded and Co-Referenced Peoplecited by 15
  27. 2021Connecting What to Say With Where to Look by Modeling Human Attention Tracescited by 13
  28. 2021Telling the What while Pointing to the Where: Multimodal Queries for Image Retrievalcited by 12
  29. 2021Coreference Resolution for the Biomedical Domain: A Surveycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025MDC-R: The Minecraft Dialogue Corpus with Referencecited by 2
  2. 2025Multimodal Coreference Resolution for Chinese Social Media Dialogues: Dataset and Benchmark Approachcited by 1
  3. 2026ReCap: Lightweight Referential Grounding for Coherent Story Visualizationcited by 1
  4. 2025Disambiguating Reference in Visually Grounded Dialogues through Joint Modeling of Textual and Multimodal Semantic Structurescited by 0
  5. 2026One Identity, Many Roles: Multimodal Entity Coreference for Enhanced Video Situation Recognitioncited by 0
  6. 2026Plug-and-Adapt: Multimodal Coreference Resolution at First Sight with a Pretrained Alignment Modelcited by 0
  7. 2026Vision-Language Grounding as Bidirectional Concept Correspondencecited by 0