# Fairness in Visual Clustering: A Novel Transformer Clustering Approach

arXiv: https://arxiv.org/abs/2304.07408 · HTML: https://mlsanity.com/paper/2304.07408/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 56 references that are in the map.
- First posted on arXiv 14 Apr 2023.

## Builds on (most cited first, 30 of 39)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference](https://mlsanity.com/paper/2104.01136/) (2021)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning](https://mlsanity.com/paper/1911.10692/) (2019)
- [End-to-End Human Object Interaction Detection with HOI Transformer](https://mlsanity.com/paper/2103.04503/) (2021)
- [Face Recognition: Too Bias, or Not Too Bias?](https://mlsanity.com/paper/2002.06483/) (2020)
- [Towards Measuring Fairness in AI: the Casual Conversations Dataset](https://mlsanity.com/paper/2104.02821/) (2021)
- [Diversity in Faces](https://mlsanity.com/paper/1901.10436/) (2019)
- [Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels](https://mlsanity.com/paper/1512.06974/) (2015)
- [Meta Balanced Network for Fair Face Recognition](https://mlsanity.com/paper/2205.06548/) (2022)
- [Learning to Cluster Faces on an Affinity Graph](https://mlsanity.com/paper/1904.02749/) (2019)
- [Understanding and Mitigating Annotation Bias in Facial Expression Recognition](https://mlsanity.com/paper/2108.08504/) (2021)
- [Vision Transformer with Progressive Sampling](https://mlsanity.com/paper/2108.01684/) (2021)
- [Micron-BERT: BERT-based Facial Micro-Expression Recognition](https://mlsanity.com/paper/2304.03195/) (2023)
- [Learning to Cluster Faces via Confidence and Connectivity Estimation](https://mlsanity.com/paper/2004.00445/) (2020)
- [Consistent Instance False Positive Improves Fairness in Face Recognition](https://mlsanity.com/paper/2106.05519/) (2021)
- [Learning Fair Classifiers with Partially Annotated Group Labels](https://mlsanity.com/paper/2111.14581/) (2021)
- [Measuring Representational Harms in Image Captioning](https://mlsanity.com/paper/2206.07173/) (2022)
- [Face Quality Estimation and Its Correlation to Demographic and Non-Demographic Bias in Face Recognition](https://mlsanity.com/paper/2004.01019/) (2020)
- [FREDOM: Fairness Domain Adaptation Approach to Semantic Scene Understanding](https://mlsanity.com/paper/2304.02135/) (2023)

## Built on by (most cited first, 10 of 10)

- [Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding](https://mlsanity.com/paper/2311.15206/) (2023)
- [QClusformer: A Quantum Transformer-based Framework for Unsupervised Visual Clustering](https://mlsanity.com/paper/2405.19722/) (2024)
- [Quantum Visual Feature Encoding Revisited](https://mlsanity.com/paper/2405.19725/) (2024)
- [Hierarchical Quantum Control Gates for Functional MRI Understanding](https://mlsanity.com/paper/2408.03596/) (2024)
- [Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding](https://mlsanity.com/paper/2502.09906/) (2025)
- [BRACTIVE: A Brain Activation Approach to Human Visual Brain Learning](https://mlsanity.com/paper/2405.18808/) (2024)
- [COBRA: A Continual Learning Approach to Vision-Brain Understanding](https://mlsanity.com/paper/2411.17475/) (2024)
- [Video-Based Autism Detection with Deep Learning](https://mlsanity.com/paper/2402.16774/) (2024)
- [A Novel Dataset for Video-Based Neurodivergent Classification Leveraging Extra-Stimulatory Behavior](https://mlsanity.com/paper/2409.04598/) (2024)
- [BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understanding](https://mlsanity.com/paper/2508.18187/) (2025)

## Abstract (arXiv)

Promoting fairness for deep clustering models in unsupervised clustering settings to reduce demographic bias is a challenging goal. This is because of the limitation of large-scale balanced data with well-annotated labels for sensitive or protected attributes. In this paper, we first evaluate demographic bias in deep clustering models from the perspective of cluster purity, which is measured by the ratio of positive samples within a cluster to their correlation degree. This measurement is adopted as an indication of demographic bias. Then, a novel loss function is introduced to encourage a purity consistency for all clusters to maintain the fairness aspect of the learned clustering model. Moreover, we present a novel attention mechanism, Cross-attention, to measure correlations between multiple clusters, strengthening faraway positive samples and improving the purity of clusters during the learning process. Experimental results on a large-scale dataset with numerous attribute settings have demonstrated the effectiveness of the proposed approach on both clustering accuracy and fairness enhancement on several sensitive attributes.