mlsanityData to 14 Oct 2026

Model-Agnostic Gender Debiased Image Captioning

arXiv 2304.03693 ↗First posted 7 Apr 2023cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Image captioning models are known to perpetuate and amplify harmful societal bias in the training set. In this work, we aim to mitigate such gender bias in image captioning models. While prior work has addressed this problem by forcing models to focus on people to reduce gender misclassification, it conversely generates gender-stereotypical words at the expense of predicting the correct gender. From this observation, we hypothesize that there are two types of gender bias affecting image captioning models: 1) bias that exploits context to predict gender, and 2) bias in the probability of generating certain (often stereotypical) words because of gender. To mitigate both types of gender biases, we propose a framework, called LIBRA, that learns from synthetically biased samples to decrease both types of biases, correcting gender misclassification and changing gender-stereotypical words to more neutral ones. Code is available at https://github.com/rebnej/LIBRA.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 56 of its 70 references that are in the map. First posted on arXiv 7 Apr 2023.

Builds on

56 of 70 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  10. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  11. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  12. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
38 more
  1. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  2. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  3. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  4. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  5. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  6. 2016Self-critical Sequence Training for Image Captioningcited by 547
  7. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  8. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  9. 2019Attention on Attention for Image Captioningcited by 289
  10. 2016Image Captioning with Semantic Attentioncited by 260
  11. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  12. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  13. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  14. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  15. 2019Does Object Recognition Work for Everyone?cited by 151
  16. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  17. 2019Fair Generative Modeling via Weak Supervisioncited by 114
  18. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  19. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
  20. 2022Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectlycited by 89
  21. 2019Predictive Inequity in Object Detectioncited by 77
  22. 2022A Systematic Study of Bias Amplificationcited by 65
  23. 2020Measuring Social Biases in Grounded Vision and Language Embeddingscited by 61
  24. 2022GRIT: Faster and Better Image captioning Transformer Using Dual Visual Featurescited by 56
  25. 2021Directional Bias Amplificationcited by 54
  26. 2021Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Modelscited by 52
  27. 2023Uncurated Image-Text Datasets: Shedding Light on Demographic Biascited by 52
  28. 2020Mitigating Gender Bias in Captioning Systemscited by 50
  29. 2022Gender and Racial Bias in Visual Question Answering Datasetscited by 39
  30. 2022Quantifying Societal Bias Amplification in Image Captioningcited by 37
  31. 2019Exposing and Correcting the Gender Bias in Image Captioning Datasets and Modelscited by 36
  32. 2021One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Visioncited by 33
  33. 2018Feature-Wise Bias Amplificationcited by 31
  34. 2022Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Modelscited by 28
  35. 2020Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lensescited by 27
  36. 2020Mitigating Gender Bias Amplification in Distribution by Posterior Regularizationcited by 27
  37. 2022Men Also Do Laundry: Multi-Attribute Bias Amplificationcited by 21
  38. 2020Show, Edit and Tell: A Framework for Editing Image Captionscited by 18

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 4; 2026: 3.

17 papers.

Sort:

By area:

  1. 2024SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIPcited by 19
  2. 2024A Unified Debiasing Approach for Vision-Language Models across Modalities and Taskscited by 18
  3. 2025A Large Scale Analysis of Gender Biases in Text-to-Image Generative Modelscited by 14
  4. 2024Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapsecited by 13
  5. 2024Fairness and Bias Mitigation in Computer Vision: A Surveycited by 13
  6. 2024From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichmentcited by 8
  7. 2024Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributescited by 7
  8. 2025Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Modelscited by 7
  9. 2024The Bias of Harmful Label Associations in Vision-Language Modelscited by 5
  10. 2025SynC: Synthetic Image Caption Dataset Refinement with One-to-many Mapping for Zero-shot Image Captioningcited by 3
  11. 2024Language-guided Detection and Mitigation of Unknown Dataset Biascited by 2
  12. 2025LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferencescited by 2
5 more
  1. 2024Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysiscited by 1
  2. 2024Think Before You Act: A Two-Stage Framework for Mitigating Gender Bias Towards Vision-Language Taskscited by 0
  3. 2026Brazilian Portuguese Image Captioning with Transformers: A Study on Cross-Native-Translated Datasetcited by 0
  4. 2026Selective Test-Time Debiasing for CLIP via Reward Gatingcited by 0
  5. 2026Anchoring Bias: A Persistent Fairness Backdoor Attack against MLLMs under Continual Learningcited by 0