# Learning from Heterogeneity: Generalizing Dynamic Facial Expression Recognition via Distributionally Robust Optimization

arXiv: https://arxiv.org/abs/2507.15765 · HTML: https://mlsanity.com/paper/2507.15765/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Builds on (most cited first, 22 of 22)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [FcaNet: Frequency Channel Attention Networks](https://mlsanity.com/paper/2012.11879/) (2020)
- [Variance-based regularization with convex objectives](https://mlsanity.com/paper/1610.02581/) (2016)
- [DFEW: A Large-Scale Database for Recognizing Dynamic Facial Expressions in the Wild](https://mlsanity.com/paper/2008.05924/) (2020)
- [FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videos](https://mlsanity.com/paper/2203.09463/) (2022)
- [Contrastive Positive Sample Propagation along the Audio-Visual Event Line](https://mlsanity.com/paper/2211.09980/) (2022)
- [Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion](https://mlsanity.com/paper/2103.16854/) (2021)
- [Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2208.10335/) (2022)
- [Spatial-Temporal Recurrent Neural Network for Emotion Recognition](https://mlsanity.com/paper/1705.04515/) (2017)
- [FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs](https://mlsanity.com/paper/2407.02157/) (2024)
- [Spatio-Temporal Transformer for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2205.04749/) (2022)
- [Distributionally Robust Models with Parametric Likelihood Ratios](https://mlsanity.com/paper/2204.06340/) (2022)
- [Sampling-Decomposable Generative Adversarial Recommender](https://mlsanity.com/paper/2011.00956/) (2020)
- [Improving Visual Prompt Tuning by Gaussian Neighborhood Minimization for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2410.21042/) (2024)
- [Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Grounding](https://mlsanity.com/paper/2403.14174/) (2024)
- [FacialPulse: An Efficient RNN-based Depression Detection via Temporal Facial Landmarks](https://mlsanity.com/paper/2408.03499/) (2024)
- [The discrete moment problem with nonconvex shape constraints](https://mlsanity.com/paper/1708.02079/) (2017)
- [Coresets for Wasserstein Distributionally Robust Optimization Problems](https://mlsanity.com/paper/2210.04260/) (2022)
- [Open-Set Video-based Facial Expression Recognition with Human Expression-sensitive Prompting](https://mlsanity.com/paper/2404.17100/) (2024)

## Built on by (most cited first, 19 of 19)

- [BalDRO: A Distributionally Robust Optimization based Framework for Large Language Model Unlearning](https://mlsanity.com/paper/2601.09172/) (2026)
- [The Cultural Gene of Large Language Models: A Study on the Impact of Cross-Corpus Training on Model Values and Biases](https://mlsanity.com/paper/2508.12411/) (2025)
- [Verifiable Fine-Tuning for LLMs: Zero-Knowledge Training Proofs Bound to Data Provenance and Policy](https://mlsanity.com/paper/2510.16830/) (2025)
- [Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning](https://mlsanity.com/paper/2507.19855/) (2025)
- [DySK-Attn: A Framework for Efficient, Real-Time Knowledge Updating in Large Language Models via Dynamic Sparse Knowledge Attention](https://mlsanity.com/paper/2508.07185/) (2025)
- [Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal](https://mlsanity.com/paper/2509.01455/) (2025)
- [Every Subtlety Counts: Fine-grained Person Independence Micro-Action Recognition via Distributionally Robust Optimization](https://mlsanity.com/paper/2509.21261/) (2025)
- [Robust Low-Rank Sparse Framework for Video-Based Affective Computing](https://mlsanity.com/paper/2511.11406/) (2025)
- [Computational Economics in Large Language Models: Exploring Model Behavior and Incentive Design under Resource Constraints](https://mlsanity.com/paper/2508.10426/) (2025)
- [MICACL: Multi-Instance Category-Aware Contrastive Learning for Long-Tailed Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2509.04344/) (2025)
- [Generalizing WiFi Gesture Recognition via Large-Model-Aware Semantic Distillation and Alignment](https://mlsanity.com/paper/2510.13390/) (2025)
- [Socialized Learning and Emergent Behaviors in Multi-Agent Systems based on Multimodal Large Language Models](https://mlsanity.com/paper/2510.18515/) (2025)
- [CoSense-LLM: Semantics at the Edge with Cost- and Uncertainty-Aware Cloud-Edge Cooperation](https://mlsanity.com/paper/2510.19670/) (2025)
- [ZK-SenseLM: Verifiable Large-Model Wireless Sensing with Selective Abstention and Zero-Knowledge Attestation](https://mlsanity.com/paper/2510.25677/) (2025)
- [Parameter-Efficient and Personalized Federated Training of Generative Models at the Edge](https://mlsanity.com/paper/2511.11585/) (2025)
- [ReMA: A Training-Free Plug-and-Play Mixing Augmentation for Video Behavior Recognition](https://mlsanity.com/paper/2601.00311/) (2026)
- [Dual-Path Learning based on Frequency Structural Decoupling and Regional-Aware Fusion for Low-Light Image Super-Resolution](https://mlsanity.com/paper/2603.27301/) (2026)
- [Quantifying Multimodal Capabilities: Formal Generalization Guarantees in Pairwise Metric Learning](https://mlsanity.com/paper/2605.01424/) (2026)
- [AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations](https://mlsanity.com/paper/2608.14130/) (2026)

## Abstract (arXiv)

Dynamic Facial Expression Recognition (DFER) plays a critical role in affective computing and human-computer interaction. Although existing methods achieve comparable performance, they inevitably suffer from performance degradation under sample heterogeneity caused by multi-source data and individual expression variability. To address these challenges, we propose a novel framework, called Heterogeneity-aware Distributional Framework (HDF), and design two plug-and-play modules to enhance time-frequency modeling and mitigate optimization imbalance caused by hard samples. Specifically, the Time-Frequency Distributional Attention Module (DAM) captures both temporal consistency and frequency robustness through a dual-branch attention design, improving tolerance to sequence inconsistency and visual style shifts. Then, based on gradient sensitivity and information bottleneck principles, an adaptive optimization module Distribution-aware Scaling Module (DSM) is introduced to dynamically balance classification and contrastive losses, enabling more stable and discriminative representation learning. Extensive experiments on two widely used datasets, DFEW and FERV39k, demonstrate that HDF significantly improves both recognition accuracy and robustness. Our method achieves superior weighted average recall (WAR) and unweighted average recall (UAR) while maintaining strong generalization across diverse and imbalanced scenarios. Codes are released at https://github.com/QIcita/HDF_DFER.