# YinYang-Net: Complementing Face and Body Information for Wild Gender Recognition

arXiv: https://arxiv.org/abs/2107.06847 · HTML: https://mlsanity.com/paper/2107.06847/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition](https://mlsanity.com/paper/1603.01249/) (2016)
- [Improving Person Re-identification by Attribute and Identity Learning](https://mlsanity.com/paper/1703.07220/) (2017)
- [HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis](https://mlsanity.com/paper/1709.09930/) (2017)
- [Pose Flow: Efficient Online Pose Tracking](https://mlsanity.com/paper/1802.00977/) (2018)
- [Deep Imbalanced Attribute Classification using Visual Attention Aggregation](https://mlsanity.com/paper/1807.03903/) (2018)
- [Improving Pedestrian Attribute Recognition With Weakly-Supervised Multi-Scale Attribute-Specific Localization](https://mlsanity.com/paper/1910.04562/) (2019)
- [Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach](https://mlsanity.com/paper/1706.00906/) (2017)
- [A Richly Annotated Dataset for Pedestrian Attribute Recognition](https://mlsanity.com/paper/1603.07054/) (2016)
- [Attribute Recognition by Joint Recurrent Learning of Context and Correlation](https://mlsanity.com/paper/1709.08553/) (2017)
- [Localization Guided Learning for Pedestrian Attribute Recognition](https://mlsanity.com/paper/1808.09102/) (2018)
- [EXTD: Extremely Tiny Face Detector via Iterative Filter Reuse](https://mlsanity.com/paper/1906.06579/) (2019)
- [Understanding and Comparing Deep Neural Networks for Age and Gender Classification](https://mlsanity.com/paper/1708.07689/) (2017)
- [Rethinking of Pedestrian Attribute Recognition: Realistic Datasets with Efficient Method](https://mlsanity.com/paper/2005.11909/) (2020)
- [Hierarchical Feature Embedding for Attribute Recognition](https://mlsanity.com/paper/2005.11576/) (2020)
- [Age Group and Gender Estimation in the Wild with Deep RoR Architecture](https://mlsanity.com/paper/1710.02985/) (2017)
- [Multimodal Age and Gender Classification Using Ear and Profile Face Images](https://mlsanity.com/paper/1907.10081/) (2019)
- [Deep Template Matching for Pedestrian Attribute Recognition with the Auxiliary Supervision of Attribute-wise Keypoints](https://mlsanity.com/paper/2011.06798/) (2020)
- [Is Gender "In-the-Wild" Inference Really a Solved Problem?](https://mlsanity.com/paper/2105.05794/) (2021)

## Built on by (most cited first, 2 of 2)

- [WASD: A Wilder Active Speaker Detection Dataset](https://mlsanity.com/paper/2303.05321/) (2023)
- [ASDnB: Merging Face with Body Cues For Robust Active Speaker Detection](https://mlsanity.com/paper/2412.08594/) (2024)

## Abstract (arXiv)

Soft biometrics inference in surveillance scenarios is a topic of interest for various applications, particularly in security-related areas. However, soft biometric analysis is not extensively reported in wild conditions. In particular, previous works on gender recognition report their results in face datasets, with relatively good image quality and frontal poses. Given the uncertainty of the availability of the facial region in wild conditions, we consider that these methods are not adequate for surveillance settings. To overcome these limitations, we: 1) present frontal and wild face versions of three well-known surveillance datasets; and 2) propose YinYang-Net (YY-Net), a model that effectively and dynamically complements facial and body information, which makes it suitable for gender recognition in wild conditions. The frontal and wild face datasets derive from widely used Pedestrian Attribute Recognition (PAR) sets (PETA, PA-100K, and RAP), using a pose-based approach to filter the frontal samples and facial regions. This approach retrieves the facial region of images with varying image/subject conditions, where the state-of-the-art face detectors often fail. YY-Net combines facial and body information through a learnable fusion matrix and a channel-attention sub-network, focusing on the most influential body parts according to the specific image/subject features. We compare it with five PAR methods, consistently obtaining state-of-the-art results on gender recognition, and reducing the prediction errors by up to 24% in frontal samples. The announced PAR datasets versions and YY-Net serve as the basis for wild soft biometrics classification and are available in https://github.com/Tiago-Roxo.