# End-to-End Facial Expression Detection in Long Videos

arXiv: https://arxiv.org/abs/2504.07660 · HTML: https://mlsanity.com/paper/2504.07660/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 10 Apr 2025.

## Builds on (most cited first, 13 of 13)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [SAMM Long Videos: A Spontaneous Facial Micro- and Macro-Expressions Dataset](https://mlsanity.com/paper/1911.01519/) (2019)
- [Shallow Optical Flow Three-Stream CNN for Macro- and Micro-Expression Spotting from Long Videos](https://mlsanity.com/paper/2106.06489/) (2021)
- [AU-aware graph convolutional network for Macro- and Micro-expression spotting](https://mlsanity.com/paper/2303.09114/) (2023)
- [Spotting Micro-Expressions on Long Videos Sequences](https://mlsanity.com/paper/1812.10306/) (2018)
- [RMES: Real-Time Micro-Expression Spotting Using Phase From Riesz Pyramid](https://mlsanity.com/paper/2305.05523/) (2023)
- [SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spotting](https://mlsanity.com/paper/2407.20799/) (2024)
- [Integrating Holistic and Local Information to Estimate Emotional Reaction Intensity](https://mlsanity.com/paper/2305.05534/) (2023)
- [Survey of Design Paradigms for Social Robots](https://mlsanity.com/paper/2407.20556/) (2024)

## Abstract (arXiv)

Facial expression detection requires spotting when expressions occur and recognizing which emotional category they belong to. Despite their close relationships, existing approaches typically address these tasks separately, limiting performance and robustness in real-world settings. In this work, we propose FEDN, a Facial Expression Detection Network, which unifies spotting and recognition into a single detection task performed fully end-to-end. FEDN introduces two temporal attention modules, segment-level attention to capture fine-grained local dynamics and sliding window attention to capture the broader temporal context. Their output is combined in a multi-scale temporal feature pyramid, which enables spotting of expressions with varying duration. This unified framework enables joint optimization and shared representation learning across tasks. FEDN outperforms strong baselines in both spotting and detection on three public benchmarks, demonstrating the effectiveness of unifying spotting and recognition across multiple temporal scales. Additionally, we uncover a previously unreported discrepancy between expert-annotated and self-reported emotion labels, highlighting a key challenge in expression benchmarking and motivating the development of more nuanced annotation protocols.