mlsanityData to 14 Oct 2026

Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels

arXiv 1512.06974 ↗First posted 22 Dec 2015cs.CVcited by 64

Not on the map. Why?

Abstract (arXiv)

When human annotators are given a choice about what to label in an image, they apply their own subjective judgments on what to ignore and what to mention. We refer to these noisy "human-centric" annotations as exhibiting human reporting bias. Examples of such annotations include image tags and keywords found on photo sharing sites, or in datasets containing image captions. In this paper, we use these noisy annotations for learning visually correct image classifiers. Such annotations do not use consistent vocabulary, and miss a significant amount of the information present in an image; however, we demonstrate that the noise in these annotations exhibits structure and can be modeled. We propose an algorithm to decouple the human reporting bias from the correct visually grounded labels. Our results are highly interpretable for reporting "what's in the image" versus "what's worth saying." We demonstrate the algorithm's efficacy along a variety of metrics and datasets, including MS COCO and Yahoo Flickr 100M. We show significant improvements over traditional algorithms for both image classification and image captioning, doubling the performance of existing methods in some cases.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Dec 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 14; 2023: 11; 2024: 12; 2025: 5; 2026: 5.

64 papers.

Sort:

By area:

  1. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  2. 2022DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotationscited by 102
  3. 2021Breaking the Dilemma of Medical Image-to-image Translationcited by 82
  4. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  5. 2021Recovering the Unbiased Scene Graphs from the Biased Onescited by 71
  6. 2023Object-Aware Distillation Pyramid for Open-Vocabulary Object Detectioncited by 63
  7. 2022Motion Policy Networkscited by 55
  8. 2021PASS: An ImageNet replacement for self-supervised pretraining without humanscited by 51
  9. 2021Transparent Human Evaluation for Image Captioningcited by 40
  10. 2022Visual Commonsense in Pretrained Unimodal and Multimodal Modelscited by 39
  11. 2021The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Colorcited by 32
  12. 2022FairGRAPE: Fairness-aware GRAdient Pruning mEthod for Face Attribute Classificationcited by 30
38 more
  1. 2022Underspecification in Scene Description-to-Depiction Taskscited by 30
  2. 2021Linguistic Structures as Weak Supervision for Visual Scene Graph Generationcited by 29
  3. 2021Multi-label Classification with Partial Annotations using Class-aware Selective Losscited by 25
  4. 2023A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?cited by 25
  5. 2023DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotationscited by 22
  6. 2021Not All Relations are Equal: Mining Informative Labels for Scene Graph Generationcited by 21
  7. 2023Multimodal Composite Association Score: Measuring Gender Bias in Generative Multimodal Modelscited by 21
  8. 2022NICEST: Noisy Label Correction and Training for Robust Scene Graph Generationcited by 19
  9. 2021A Study of the Generalizability of Self-Supervised Representationscited by 18
  10. 2022Unbiased Directed Object Attention Graph for Object Navigationcited by 18
  11. 2022Defending Graph Convolutional Networks against Dynamic Graph Perturbations via Bayesian Self-supervisioncited by 16
  12. 2022Hierarchical Memory Learning for Fine-Grained Scene Graph Generationcited by 15
  13. 2022Label Semantic Knowledge Distillation for Unbiased Scene Graph Generationcited by 14
  14. 2024TAI++: Text as Image for Multi-Label Image Classification by Co-Learning Transferable Promptcited by 14
  15. 2024Nonverbal Interaction Detectioncited by 14
  16. 2024Fairness and Bias Mitigation in Computer Vision: A Surveycited by 13
  17. 2022Ambiguous Images With Human Judgments for Robust Visual Event Classificationcited by 12
  18. 2023Towards Semi-supervised Learning with Non-random Missing Labelscited by 12
  19. 2022On Non-Random Missing Labels in Semi-Supervised Learningcited by 11
  20. 2025DIFFVSGG: Diffusion-Driven Online Video Scene Graph Generationcited by 11
  21. 2023Fairness in Visual Clustering: A Novel Transformer Clustering Approachcited by 10
  22. 2023Latent Class-Conditional Noise Modelcited by 9
  23. 2021Semantic Clustering based Deduction Learning for Image Recognition and Classificationcited by 8
  24. 2022Fine-Grained Prediction of Political Leaning on Social Media with Unsupervised Deep Learningcited by 8
  25. 2024Data-free Multi-label Image Recognition via LLM-powered Prompt Tuningcited by 8
  26. 2024From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichmentcited by 8
  27. 2021Human-centric Relation Segmentation: Dataset and Solutioncited by 7
  28. 2021Rethinking Crowdsourcing Annotation: Partial Annotation with Salient Labels for Multi-Label Image Classificationcited by 7
  29. 2021DeepMCAT: Large-Scale Deep Clustering for Medical Image Categorizationcited by 7
  30. 2024Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributescited by 7
  31. 2024Improving Multi-label Recognition using Class Co-Occurrence Probabilitiescited by 6
  32. 2025Unbiased Video Scene Graph Generation via Visual and Semantic Dual Debiasingcited by 6
  33. 2021MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labelscited by 5
  34. 2023Skew Class-balanced Re-weighting for Unbiased Scene Graph Generationcited by 5
  35. 2023Fairness in AI Systems: Mitigating gender bias from language-vision modelscited by 5
  36. 2024Learning Multiple Object States from Actions via Large Language Modelscited by 4
  37. 2023Taking A Closer Look at Visual Relation: Unbiased Video Scene Graph Generation with Decoupled Label Learningcited by 3
  38. 2024Active Negative Loss: A Robust Framework for Learning with Noisy Labelscited by 3

50 most cited of 64 All 64 (JSON)