# FairViT-GAN: A Hybrid Vision Transformer with Adversarial Debiasing for Fair and Explainable Facial Beauty Prediction

arXiv: https://arxiv.org/abs/2509.23859 · HTML: https://mlsanity.com/paper/2509.23859/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 28 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015)
- [SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction](https://mlsanity.com/paper/1801.06345/) (2018)
- [Label distribution based facial attractiveness computation by deep residual learning](https://mlsanity.com/paper/1609.00496/) (2016)
- [GMAR: Gradient-Driven Multi-Head Attention Rollout for Vision Transformer Interpretability](https://mlsanity.com/paper/2504.19414/) (2025)
- [Generative Pre-training for Subjective Tasks: A Diffusion Transformer-Based Framework for Facial Beauty Prediction](https://mlsanity.com/paper/2507.20363/) (2025)
- [Mamba-CNN: A Hybrid Architecture for Efficient and Accurate Facial Beauty Prediction](https://mlsanity.com/paper/2509.01431/) (2025)
- [Scale-interaction transformer: a hybrid cnn-transformer model for facial beauty prediction](https://mlsanity.com/paper/2509.05078/) (2025)
- [Ethically aligned Deep Learning: Unbiased Facial Aesthetic Prediction](https://mlsanity.com/paper/2111.05149/) (2021)
- [SynergyNet: Fusing Generative Priors and State-Space Models for Facial Beauty Prediction](https://mlsanity.com/paper/2509.17172/) (2025)

## Built on by (most cited first, 1 of 1)

- [VM-BeautyNet: A Synergistic Ensemble of Vision Transformer and Mamba for Facial Beauty Prediction](https://mlsanity.com/paper/2510.16220/) (2025)

## Abstract (arXiv)

Facial Beauty Prediction (FBP) has made significant strides with the application of deep learning, yet state-of-the-art models often exhibit critical limitations, including architectural constraints, inherent demographic biases, and a lack of transparency. Existing methods, primarily based on Convolutional Neural Networks (CNNs), excel at capturing local texture but struggle with global facial harmony, while Vision Transformers (ViTs) effectively model long-range dependencies but can miss fine-grained details. Furthermore, models trained on benchmark datasets can inadvertently learn and perpetuate societal biases related to protected attributes like ethnicity. To address these interconnected challenges, we propose FairViT-GAN, a novel hybrid framework that synergistically integrates a CNN branch for local feature extraction and a ViT branch for global context modeling. More significantly, we introduce an adversarial debiasing mechanism where the feature extractor is explicitly trained to produce representations that are invariant to protected attributes, thereby actively mitigating algorithmic bias. Our framework's transparency is enhanced by visualizing the distinct focus of each architectural branch. Extensive experiments on the SCUT-FBP5500 benchmark demonstrate that FairViT-GAN not only sets a new state-of-the-art in predictive accuracy, achieving a Pearson Correlation of 0.9230 and reducing RMSE to 0.2650, but also excels in fairness. Our analysis reveals a remarkable 82.9% reduction in the performance gap between ethnic subgroups, with the adversary's classification accuracy dropping to near-random chance (52.1%). We believe FairViT-GAN provides a robust, transparent, and significantly fairer blueprint for developing responsible AI systems for subjective visual assessment.