# Loss Function Search for Face Recognition

arXiv: https://arxiv.org/abs/2007.06542 · HTML: https://mlsanity.com/paper/2007.06542/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2020.

## Built on by (most cited first, 22 of 22)

- [Automated Self-Supervised Learning for Graphs](https://mlsanity.com/paper/2106.05470/) (2021)
- [AutoLoss: Automated Loss Function Search in Recommendations](https://mlsanity.com/paper/2106.06713/) (2021)
- [SphereFace Revived: Unifying Hyperspherical Face Recognition](https://mlsanity.com/paper/2109.05565/) (2021)
- [Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search](https://mlsanity.com/paper/2102.04700/) (2021)
- [AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks](https://mlsanity.com/paper/2103.14026/) (2021)
- [Reinforcement Learning with Automated Auxiliary Loss Search](https://mlsanity.com/paper/2210.06041/) (2022)
- [An Efficient Training Approach for Very Large Scale Face Recognition](https://mlsanity.com/paper/2105.10375/) (2021)
- [AutoLossGen: Automatic Loss Function Generation for Recommender Systems](https://mlsanity.com/paper/2204.13160/) (2022)
- [Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-free Multi-Exposure Image Fusion](https://mlsanity.com/paper/2309.01113/) (2023)
- [Toward a Better Understanding of Loss Functions for Collaborative Filtering](https://mlsanity.com/paper/2308.06091/) (2023)
- [Local Multi-Head Channel Self-Attention for Facial Expression Recognition](https://mlsanity.com/paper/2111.07224/) (2021)
- [Hyperparameter Tuning for Deep Reinforcement Learning Applications](https://mlsanity.com/paper/2201.11182/) (2022)
- [Meta-Learning Adaptive Loss Functions](https://mlsanity.com/paper/2301.13247/) (2023)
- [Meta-tuning Loss Functions and Data Augmentation for Few-shot Object Detection](https://mlsanity.com/paper/2304.12161/) (2023)
- [DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning](https://mlsanity.com/paper/2305.07892/) (2023)
- [Prototype Memory for Large-scale Face Representation Learning](https://mlsanity.com/paper/2105.02103/) (2021)
- [Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search](https://mlsanity.com/paper/2111.05063/) (2021)
- [Towards Robust Face Recognition with Comprehensive Search](https://mlsanity.com/paper/2208.13600/) (2022)
- [HLIC: Harmonizing Optimization Metrics in Learned Image Compression by Reinforcement Learning](https://mlsanity.com/paper/2109.14863/) (2021)
- [Next Generation Loss Function for Image Classification](https://mlsanity.com/paper/2404.12948/) (2024)
- [Application of Self-Supervised Learning to MICA Model for Reconstructing Imperfect 3D Facial Structures](https://mlsanity.com/paper/2304.04060/) (2023)
- [Automated Loss function Search for Class-imbalanced Node Classification](https://mlsanity.com/paper/2405.14133/) (2024)

## Abstract (arXiv)

In face recognition, designing margin-based (e.g., angular, additive, additive angular margins) softmax loss functions plays an important role in learning discriminative features. However, these hand-crafted heuristic methods are sub-optimal because they require much effort to explore the large design space. Recently, an AutoML for loss function search method AM-LFS has been derived, which leverages reinforcement learning to search loss functions during the training process. But its search space is complex and unstable that hindering its superiority. In this paper, we first analyze that the key to enhance the feature discrimination is actually how to reduce the softmax probability. We then design a unified formulation for the current margin-based softmax losses. Accordingly, we define a novel search space and develop a reward-guided search method to automatically obtain the best candidate. Experimental results on a variety of face recognition benchmarks have demonstrated the effectiveness of our method over the state-of-the-art alternatives.