mlsanityData to 7 Oct 2026

Consistency and Accuracy of CelebA Attribute Values

arXiv 2210.07356 ↗First posted 13 Oct 2022cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

We report the first systematic analysis of the experimental foundations of facial attribute classification. Two annotators independently assigning attribute values shows that only 12 of 40 common attributes are assigned values with >= 95% consistency, and three (high cheekbones, pointed nose, oval face) have essentially random consistency. Of 5,068 duplicate face appearances in CelebA, attributes have contradicting values on from 10 to 860 of the 5,068 duplicates. Manual audit of a subset of CelebA estimates error rates as high as 40% for (no beard=false), even though the labeling consistency experiment indicates that no beard could be assigned with >= 95% consistency. Selecting the mouth slightly open (MSO) for deeper analysis, we estimate the error rate for (MSO=true) at about 20% and (MSO=false) at about 2%. A corrected version of the MSO attribute values enables learning a model that achieves higher accuracy than previously reported for MSO. Corrected values for CelebA MSO are available at https://github.com/HaiyuWu/CelebAMSO.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 16 of its 36 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

16 of 36 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  6. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  7. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  8. 2017AttGAN: Facial Attribute Editing by Only Changing What You Wantcited by 241
  9. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  10. 2016MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributescited by 34
  11. 2020MAAD-Face: A Massively Annotated Attribute Dataset for Face Imagescited by 31
  12. 2016Deep Attributes Driven Multi-Camera Person Re-identificationcited by 21
4 more
  1. 2017Transferring a Semantic Representation for Person Re-Identification and Searchcited by 7
  2. 2018A Survey of Deep Facial Attribute Analysiscited by 6
  3. 2016AFFACT - Alignment-Free Facial Attribute Classification Techniquecited by 4
  4. 2020Dataset Cleaning -- A Cross Validation Methodology for Large Facial Datasets using Face Recognitioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 4; 2026: 3.

11 papers.

  1. 2023Logical Consistency and Greater Descriptive Power for Facial Hair Attribute Learningcited by 15
  2. 2025Diffusion Counterfactual Generation with Semantic Abductioncited by 8
  3. 2025Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generationcited by 5
  4. 2024Can Vision-Language Models Replace Human Annotators: A Case Study with CelebA Datasetcited by 4
  5. 2024Identity Overlap Between Face Recognition Train/Test Data: Causing Optimistic Bias in Accuracy Measurementcited by 3
  6. 2025Discovering Intersectional Bias via Directional Alignment in Face Recognition Embeddingscited by 2
  7. 2024Biased Binary Attribute Classifiers Ignore the Majority Classescited by 1
  8. 2025Bias Analysis in Unconditional Image Generative Modelscited by 0
  9. 2026Bias Leaves a Gradient Trail: Label-Free Bias Identification via Gradient Probes on Concept Decompositionscited by 0
  10. 2026Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?cited by 0
  11. 2026Do Counterfactually Fair Image Classifiers Satisfy Group Fairness? -- A Theoretical and Empirical Studycited by 0