# Facial Soft Biometrics for Recognition in the Wild: Recent Works, Annotation, and COTS Evaluation

arXiv: https://arxiv.org/abs/2210.13129 · HTML: https://mlsanity.com/paper/2210.13129/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 43 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 5 of 5)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach](https://mlsanity.com/paper/1706.00906/) (2017)
- [MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributes](https://mlsanity.com/paper/1603.07027/) (2016)
- [Quality Measures in Biometric Systems](https://mlsanity.com/paper/2111.08704/) (2021)
- [Convolutional Neural Networks for Attribute-based Active Authentication on Mobile Devices](https://mlsanity.com/paper/1604.08865/) (2016)

## Built on by (most cited first, 19 of 19)

- [Towards General Visual-Linguistic Face Forgery Detection](https://mlsanity.com/paper/2307.16545/) (2023)
- [How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainability](https://mlsanity.com/paper/2401.13641/) (2024)
- [Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment](https://mlsanity.com/paper/2302.10908/) (2023)
- [Measuring Bias in AI Models: An Statistical Approach Introducing N-Sigma](https://mlsanity.com/paper/2304.13680/) (2023)
- [Periocular Biometrics: A Modality for Unconstrained Scenarios](https://mlsanity.com/paper/2212.13792/) (2022)
- [Toward Face Biometric De-identification using Adversarial Examples](https://mlsanity.com/paper/2302.03657/) (2023)
- [OTB-morph: One-Time Biometrics via Morphing](https://mlsanity.com/paper/2302.09053/) (2023)
- [Keystroke Verification Challenge (KVC): Biometric and Fairness Benchmark Evaluation](https://mlsanity.com/paper/2311.06000/) (2023)
- [Exploiting Multiple Representations: 3D Face Biometrics Fusion with Application to Surveillance](https://mlsanity.com/paper/2504.18886/) (2025)
- [Turn Fake into Real: Adversarial Head Turn Attacks Against Deepfake Detection](https://mlsanity.com/paper/2309.01104/) (2023)
- [Improving Face Recognition from Caption Supervision with Multi-Granular Contextual Feature Aggregation](https://mlsanity.com/paper/2308.06866/) (2023)
- [Massively Annotated Datasets for Assessment of Synthetic and Real Data in Face Recognition](https://mlsanity.com/paper/2404.15234/) (2024)
- [Balancing Tails when Comparing Distributions: Comprehensive Equity Index (CEI) with Application to Bias Evaluation in Operational Face Biometrics](https://mlsanity.com/paper/2506.10564/) (2025)
- [Image Generation and Learning Strategy for Deep Document Forgery Detection](https://mlsanity.com/paper/2311.03650/) (2023)
- [Text-Guided Face Recognition using Multi-Granularity Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2312.09367/) (2023)
- [Towards Robust Protective Perturbation against DeepFake Face Swapping](https://mlsanity.com/paper/2512.07228/) (2025)
- [Towards General Visual-Linguistic Face Forgery Detection(V2)](https://mlsanity.com/paper/2502.20698/) (2025)
- [Weight and Height Estimation from a Single Human Image Captured in the Wild](https://mlsanity.com/paper/2607.26104/) (2026)
- [VideoRun2D Demo: Markerless Body Tracking for Biomechanical Analysis of Running](https://mlsanity.com/paper/2608.19480/) (2026)

## Abstract (arXiv)

The role of soft biometrics to enhance person recognition systems in unconstrained scenarios has not been extensively studied. Here, we explore the utility of the following modalities: gender, ethnicity, age, glasses, beard, and moustache. We consider two assumptions: 1) manual estimation of soft biometrics and 2) automatic estimation from two commercial off-the-shelf systems (COTS). All experiments are reported using the labeled faces in the wild (LFW) database. First, we study the discrimination capabilities of soft biometrics standalone. Then, experiments are carried out fusing soft biometrics with two state-of-the-art face recognition systems based on deep learning. We observe that soft biometrics is a valuable complement to the face modality in unconstrained scenarios, with relative improvements up to 40%/15% in the verification performance when using manual/automatic soft biometrics estimation. Results are reproducible as we make public our manual annotations and COTS outputs of soft biometrics over LFW, as well as the face recognition scores.