# Facial Attractiveness Prediction in Live Streaming: A New Benchmark and Multi-modal Method

arXiv: https://arxiv.org/abs/2501.02509 · HTML: https://mlsanity.com/paper/2501.02509/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 80 references that are in the map.
- First posted on arXiv 5 Jan 2025.

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)
- [Photo Aesthetics Ranking Network with Attributes and Content Adaptation](https://mlsanity.com/paper/1606.01621/) (2016)
- [SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness](https://mlsanity.com/paper/2003.09373/) (2020)
- [CR-FIQA: Face Image Quality Assessment by Learning Sample Relative Classifiability](https://mlsanity.com/paper/2112.06592/) (2021)
- [SDD-FIQA: Unsupervised Face Image Quality Assessment with Similarity Distribution Distance](https://mlsanity.com/paper/2103.05977/) (2021)
- [Effective Aesthetics Prediction with Multi-level Spatially Pooled Features](https://mlsanity.com/paper/1904.01382/) (2019)
- [Image Aesthetic Assessment: An Experimental Survey](https://mlsanity.com/paper/1610.00838/) (2016)
- [FaceBoxes: A CPU Real-time Face Detector with High Accuracy](https://mlsanity.com/paper/1708.05234/) (2017)
- [SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation](https://mlsanity.com/paper/2308.11509/) (2023)
- [SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction](https://mlsanity.com/paper/1801.06345/) (2018)
- [Going the Extra Mile in Face Image Quality Assessment: A Novel Database and Model](https://mlsanity.com/paper/2207.04904/) (2022)
- [CLIP-DR: Textual Knowledge-Guided Diabetic Retinopathy Grading with Ranking-aware Prompting](https://mlsanity.com/paper/2407.04068/) (2024)
- [DSL-FIQA: Assessing Facial Image Quality via Dual-Set Degradation Learning and Landmark-Guided Transformer](https://mlsanity.com/paper/2406.09622/) (2024)
- [An Image Quality Assessment Dataset for Portraits](https://mlsanity.com/paper/2304.05772/) (2023)
- [Label distribution based facial attractiveness computation by deep residual learning](https://mlsanity.com/paper/1609.00496/) (2016)
- [Dual-Branch Network for Portrait Image Quality Assessment](https://mlsanity.com/paper/2405.08555/) (2024)
- [Some like it hot - visual guidance for preference prediction](https://mlsanity.com/paper/1510.07867/) (2015)
- [Generalized Visual Quality Assessment of GAN-Generated Face Images](https://mlsanity.com/paper/2201.11975/) (2022)
- [Generalized Portrait Quality Assessment](https://mlsanity.com/paper/2402.09178/) (2024)
- [ComboLoss for Facial Attractiveness Analysis with Squeeze-and-Excitation Networks](https://mlsanity.com/paper/2010.10721/) (2020)
- [Lightweight Facial Attractiveness Prediction Using Dual Label Distribution](https://mlsanity.com/paper/2212.01742/) (2022)

## Abstract (arXiv)

Facial attractiveness prediction (FAP) has long been an important computer vision task, which could be widely applied in live streaming for facial retouching, content recommendation, etc. However, previous FAP datasets are either small, closed-source, or lack diversity. Moreover, the corresponding FAP models exhibit limited generalization and adaptation ability. To overcome these limitations, in this paper we present LiveBeauty, the first large-scale live-specific FAP dataset, in a more challenging application scenario, i.e., live streaming. 10,000 face images are collected from a live streaming platform directly, with 200,000 corresponding attractiveness annotations obtained from a well-devised subjective experiment, making LiveBeauty the largest open-access FAP dataset in the challenging live scenario. Furthermore, a multi-modal FAP method is proposed to measure the facial attractiveness in live streaming. Specifically, we first extract holistic facial prior knowledge and multi-modal aesthetic semantic features via a Personalized Attractiveness Prior Module (PAPM) and a Multi-modal Attractiveness Encoder Module (MAEM), respectively, then integrate the extracted features through a Cross-Modal Fusion Module (CMFM). Extensive experiments conducted on both LiveBeauty and other open-source FAP datasets demonstrate that our proposed method achieves state-of-the-art performance. Dataset will be available soon.