mlsanityData to 14 Oct 2026

Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributes

arXiv 2407.03623 ↗First posted 4 Jul 2024cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We tackle societal bias in image-text datasets by removing spurious correlations between protected groups and image attributes. Traditional methods only target labeled attributes, ignoring biases from unlabeled ones. Using text-guided inpainting models, our approach ensures protected group independence from all attributes and mitigates inpainting biases through data filtering. Evaluations on multi-label image classification and image captioning tasks show our method effectively reduces bias without compromising performance across various models.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 58 of its 78 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

58 of 78 references.

References · 58

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2023Segment Anythingcited by 8,234
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  10. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  12. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
38 more
  1. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  2. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 647
  3. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  4. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 439
  5. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  6. 2023DIRE for Diffusion-Generated Image Detectioncited by 390
  7. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  8. 2022On the detection of synthetic images generated by diffusion modelscited by 283
  9. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  10. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  11. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 251
  12. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  13. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  14. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  15. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 226
  16. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  17. 2019Does Object Recognition Work for Everyone?cited by 151
  18. 2023Fair Diffusion: Instructing Text-to-Image Generation Models on Fairnesscited by 151
  19. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  20. 2023Social Biases through the Text-to-Image Generation Lenscited by 127
  21. 2020Fair Attribute Classification through Latent Space De-biasingcited by 125
  22. 2018Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddingscited by 114
  23. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  24. 2022How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?cited by 92
  25. 2020Robustness to Spurious Correlations in Text Classification via Automatically Generated Counterfactualscited by 67
  26. 2015Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labelscited by 64
  27. 2020Measuring Social Biases in Grounded Vision and Language Embeddingscited by 61
  28. 2021Directional Bias Amplificationcited by 54
  29. 2023Uncurated Image-Text Datasets: Shedding Light on Demographic Biascited by 52
  30. 2020Mitigating Gender Bias in Captioning Systemscited by 50
  31. 2021A Step Toward More Inclusive People Annotations for Fairnesscited by 46
  32. 2020Gender Slopes: Counterfactual Fairness for Computer Vision Models by Attribute Manipulationcited by 37
  33. 2016Faceless Person Recognition; Privacy Implications in Social Mediacited by 27
  34. 2023Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Setscited by 27
  35. 2023Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Modelscited by 27
  36. 2017Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Imagescited by 24
  37. 2023Auditing Gender Presentation Differences in Text-to-Image Modelscited by 22
  38. 2023Consensus and Subjectivity of Skin Tone Annotation for ML Fairnesscited by 22

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 1.

7 papers.

  1. 2024SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIPcited by 19
  2. 2026Shortcut Learning in Binary Classifier Black Boxes: Applications to Voice Anti-Spoofing and Biometricscited by 6
  3. 2025Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluationcited by 5
  4. 2025Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Modelscited by 4
  5. 2025SPICE: A Synergistic, Precise, Iterative, and Customizable Image Editing Workflowcited by 2
  6. 2025LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferencescited by 2
  7. 2025Code-Driven Inductive Synthesis: Enhancing Reasoning Abilities of Large Language Models with Sequencescited by 1