mlsanityData to 7 Oct 2026

Deconfounded Image Captioning: A Causal Retrospect

arXiv 2003.03923 ↗First posted 9 Mar 2020cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Dataset bias in vision-language tasks is becoming one of the main problems which hinders the progress of our community. Existing solutions lack a principled analysis about why modern image captioners easily collapse into dataset bias. In this paper, we present a novel perspective: Deconfounded Image Captioning (DIC), to find out the answer of this question, then retrospect modern neural image captioners, and finally propose a DIC framework: DICv1.0 to alleviate the negative effects brought by dataset bias. DIC is based on causal inference, whose two principles: the backdoor and front-door adjustments, help us review previous studies and design new effective models. In particular, we showcase that DICv1.0 can strengthen two prevailing captioning models and can achieve a single-model 131.1 CIDEr-D and 128.4 c40 CIDEr-D on Karpathy split and online split of the challenging MS COCO dataset, respectively. Interestingly, DICv1.0 is a natural derivation from our causal retrospect, which opens promising directions for image captioning.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 12; 2023: 13; 2024: 8; 2025: 12; 2026: 3.

65 papers.

Sort:

By area:

  1. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  2. 2021Counterfactual Zero-Shot and Open-Set Visual Recognitioncited by 95
  3. 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
  4. 2021Causal Attention for Unbiased Visual Recognitioncited by 80
  5. 2021Let there be a clock on the beach: Reducing Object Hallucination in Image Captioningcited by 74
  6. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  7. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  8. 2021Frustratingly Easy Transferability Estimationcited by 44
  9. 2024Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustmentcited by 29
  10. 2021Consensus Graph Representation Learning for Better Grounded Image Captioningcited by 28
  11. 2021MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioningcited by 27
  12. 2022Causal Reasoning Meets Visual Representation Learning: A Prospective Studycited by 24
38 more
  1. 2022Open Vocabulary Object Detection with Proposal Mining and Prediction Equalizationcited by 23
  2. 2024MECD: Unlocking Multi-Event Causal Discovery in Video Reasoningcited by 23
  3. 2021Learning Domain Invariant Representations for Generalizable Person Re-Identificationcited by 22
  4. 2021Uncovering Main Causalities for Long-tailed Information Extractioncited by 21
  5. 2021Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classificationcited by 19
  6. 2023Causal Unsupervised Semantic Segmentationcited by 18
  7. 2021Deconfounded Visual Groundingcited by 16
  8. 2023Visual Causal Scene Refinement for Video Question Answeringcited by 16
  9. 2021Improving Weakly-supervised Object Localization via Causal Interventioncited by 14
  10. 2023Cross-Modal Causal Intervention for Medical Report Generationcited by 14
  11. 2021Weakly-Supervised Video Object Grounding via Causal Interventioncited by 13
  12. 2023Causal reasoning in typical computer vision taskscited by 12
  13. 2021Towards Unbiased Visual Emotion Recognition via Causal Interventioncited by 11
  14. 2022A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approachcited by 11
  15. 2023Simple Token-Level Confidence Improves Caption Correctnesscited by 11
  16. 2021Conterfactual Generative Zero-Shot Semantic Segmentationcited by 10
  17. 2024Debiased Multimodal Understanding for Human Language Sequencescited by 10
  18. 2021DROID: Driver-centric Risk Object Identificationcited by 9
  19. 2025CleanPose: Category-Level Object Pose Estimation via Causal Learning and Knowledge Distillationcited by 9
  20. 2022Causal Inference via Style Transfer for Out-of-distribution Generalisationcited by 8
  21. 2023Hacking Task Confounder in Meta-Learningcited by 8
  22. 2022Image Captioning via Compact Bidirectional Architecturecited by 7
  23. 2025Resilient Contrastive Pre-training under Non-Stationary Driftcited by 7
  24. 2022What's in a Caption? Dataset-Specific Linguistic Diversity and Its Effect on Visual Description Models and Metricscited by 6
  25. 2022Distribution Aware Metrics for Conditional Natural Language Generationcited by 6
  26. 2025Structure Causal Models and LLMs Integration in Medical Visual Question Answeringcited by 6
  27. 2023ImageCaptioner2: Image Captioner for Image Captioning Bias Amplification Assessmentcited by 5
  28. 2023Mitigating Prior Errors in Causal Structure Learning: A Resilient Approach via Bayesian Networkscited by 5
  29. 2023Backdoor Adjustment of Confounding by Provenance for Robust Text Classification of Multi-institutional Clinical Notescited by 5
  30. 2022Revisiting Few-Shot Learning from a Causal Perspectivecited by 4
  31. 2024Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysiscited by 4
  32. 2025Walking the Tightrope: Disentangling Beneficial and Detrimental Drifts in Non-Stationary Custom-Tuningcited by 4
  33. 2025Causal-LLaVA: Causal Disentanglement for Mitigating Hallucination in Multimodal Large Language Modelscited by 4
  34. 2022Image Captioning In the Transformer Agecited by 3
  35. 2023VCD: Visual Causality Discovery for Cross-Modal Question Reasoningcited by 2
  36. 2023Instrumental Variable Learning for Chest X-ray Classificationcited by 2
  37. 2025EQ-TAA: Equivariant Traffic Accident Anticipation via Diffusion-Based Accident Video Synthesiscited by 2
  38. 2025Causal-Entity Reflected Egocentric Traffic Accident Video Synthesiscited by 2

50 most cited of 65 All 65 (JSON)