# Consistency and Accuracy of CelebA Attribute Values

arXiv: https://arxiv.org/abs/2210.07356 · HTML: https://mlsanity.com/paper/2210.07356/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributes](https://mlsanity.com/paper/1603.07027/) (2016)
- [MAAD-Face: A Massively Annotated Attribute Dataset for Face Images](https://mlsanity.com/paper/2012.01030/) (2020)
- [Deep Attributes Driven Multi-Camera Person Re-identification](https://mlsanity.com/paper/1605.03259/) (2016)
- [Transferring a Semantic Representation for Person Re-Identification and Search](https://mlsanity.com/paper/1706.03725/) (2017)
- [A Survey of Deep Facial Attribute Analysis](https://mlsanity.com/paper/1812.10265/) (2018)
- [AFFACT - Alignment-Free Facial Attribute Classification Technique](https://mlsanity.com/paper/1611.06158/) (2016)
- [Dataset Cleaning -- A Cross Validation Methodology for Large Facial Datasets using Face Recognition](https://mlsanity.com/paper/2003.10815/) (2020)

## Built on by (most cited first, 11 of 11)

- [Logical Consistency and Greater Descriptive Power for Facial Hair Attribute Learning](https://mlsanity.com/paper/2302.11102/) (2023)
- [Diffusion Counterfactual Generation with Semantic Abduction](https://mlsanity.com/paper/2506.07883/) (2025)
- [Can Vision-Language Models Replace Human Annotators: A Case Study with CelebA Dataset](https://mlsanity.com/paper/2410.09416/) (2024)
- [Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generation](https://mlsanity.com/paper/2508.10672/) (2025)
- [Identity Overlap Between Face Recognition Train/Test Data: Causing Optimistic Bias in Accuracy Measurement](https://mlsanity.com/paper/2405.09403/) (2024)
- [Discovering Intersectional Bias via Directional Alignment in Face Recognition Embeddings](https://mlsanity.com/paper/2510.15520/) (2025)
- [Biased Binary Attribute Classifiers Ignore the Majority Classes](https://mlsanity.com/paper/2403.14435/) (2024)
- [Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?](https://mlsanity.com/paper/2605.30642/) (2026)
- [Do Counterfactually Fair Image Classifiers Satisfy Group Fairness? -- A Theoretical and Empirical Study](https://mlsanity.com/paper/2607.06603/) (2026)
- [Bias Analysis in Unconditional Image Generative Models](https://mlsanity.com/paper/2506.09106/) (2025)
- [Bias Leaves a Gradient Trail: Label-Free Bias Identification via Gradient Probes on Concept Decompositions](https://mlsanity.com/paper/2605.28780/) (2026)

## Abstract (arXiv)

We report the first systematic analysis of the experimental foundations of facial attribute classification. Two annotators independently assigning attribute values shows that only 12 of 40 common attributes are assigned values with >= 95% consistency, and three (high cheekbones, pointed nose, oval face) have essentially random consistency. Of 5,068 duplicate face appearances in CelebA, attributes have contradicting values on from 10 to 860 of the 5,068 duplicates. Manual audit of a subset of CelebA estimates error rates as high as 40% for (no beard=false), even though the labeling consistency experiment indicates that no beard could be assigned with >= 95% consistency. Selecting the mouth slightly open (MSO) for deeper analysis, we estimate the error rate for (MSO=true) at about 20% and (MSO=false) at about 2%. A corrected version of the MSO attribute values enables learning a model that achieves higher accuracy than previously reported for MSO. Corrected values for CelebA MSO are available at https://github.com/HaiyuWu/CelebAMSO.