# Multi-threshold Deep Metric Learning for Facial Expression Recognition

arXiv: https://arxiv.org/abs/2406.16434 · HTML: https://mlsanity.com/paper/2406.16434/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 70 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 28 of 28)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Cross-Batch Memory for Embedding Learning](https://mlsanity.com/paper/1912.06798/) (2019)
- [Deep Metric Learning with Hierarchical Triplet Loss](https://mlsanity.com/paper/1810.06951/) (2018)
- [Deep Metric Learning with Angular Loss](https://mlsanity.com/paper/1708.01682/) (2017)
- [Smart Mining for Deep Metric Learning](https://mlsanity.com/paper/1704.01285/) (2017)
- [Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition](https://mlsanity.com/paper/2104.00232/) (2021)
- [Attention-based Ensemble for Deep Metric Learning](https://mlsanity.com/paper/1804.00382/) (2018)
- [Hard-Aware Deeply Cascaded Embedding](https://mlsanity.com/paper/1611.05720/) (2016)
- [Deep Metric Learning with BIER: Boosting Independent Embeddings Robustly](https://mlsanity.com/paper/1801.04815/) (2018)
- [Covariance Pooling For Facial Expression Recognition](https://mlsanity.com/paper/1805.04855/) (2018)
- [Island Loss for Learning Discriminative Features in Facial Expression Recognition](https://mlsanity.com/paper/1710.03144/) (2017)
- [MixAugment & Mixup: Augmentation Methods for Facial Expression Recognition](https://mlsanity.com/paper/2205.04442/) (2022)
- [Embedding Expansion: Augmentation in Embedding Space for Deep Metric Learning](https://mlsanity.com/paper/2003.02546/) (2020)
- [Hard Negative Mining for Metric Learning Based Zero-Shot Classification](https://mlsanity.com/paper/1608.07441/) (2016)
- [Sharing Matters for Generalization in Deep Metric Learning](https://mlsanity.com/paper/2004.05582/) (2020)
- [Identity-Free Facial Expression Recognition using conditional Generative Adversarial Network](https://mlsanity.com/paper/1903.08051/) (2019)
- [The FaceChannel: A Fast & Furious Deep Neural Network for Facial Expression Recognition](https://mlsanity.com/paper/2009.07635/) (2020)
- [Cross-Cultural and Cultural-Specific Production and Perception of Facial Expressions of Emotion in the Wild](https://mlsanity.com/paper/1808.04399/) (2018)
- [LBVCNN: Local Binary Volume Convolutional Neural Network for Facial Expression Recognition from Image Sequences](https://mlsanity.com/paper/1904.07647/) (2019)
- [The FaceChannel: A Light-weight Deep Neural Network for Facial Expression Recognition](https://mlsanity.com/paper/2004.08195/) (2020)

## Built on by (most cited first, 1 of 1)

- [Context-Aware Academic Emotion Dataset and Benchmark](https://mlsanity.com/paper/2507.00586/) (2025)

## Abstract (arXiv)

Effective expression feature representations generated by a triplet-based deep metric learning are highly advantageous for facial expression recognition (FER). The performance of triplet-based deep metric learning is contingent upon identifying the best threshold for triplet loss. Threshold validation, however, is tough and challenging, as the ideal threshold changes among datasets and even across classes within the same dataset. In this paper, we present the multi-threshold deep metric learning technique, which not only avoids the difficult threshold validation but also vastly increases the capacity of triplet loss learning to construct expression feature representations. We find that each threshold of the triplet loss intrinsically determines a distinctive distribution of inter-class variations and corresponds, thus, to a unique expression feature representation. Therefore, rather than selecting a single optimal threshold from a valid threshold range, we thoroughly sample thresholds across the range, allowing the representation characteristics manifested by thresholds within the range to be fully extracted and leveraged for FER. To realize this approach, we partition the embedding layer of the deep metric learning network into a collection of slices and model training these embedding slices as an end-to-end multi-threshold deep metric learning problem. Each embedding slice corresponds to a sample threshold and is learned by enforcing the corresponding triplet loss, yielding a set of distinct expression features, one for each embedding slice. It makes the embedding layer, which is composed of a set of slices, a more informative and discriminative feature, hence enhancing the FER accuracy. Extensive evaluations demonstrate the superior performance of the proposed approach on both posed and spontaneous facial expression datasets.