# FER-YOLO-Mamba: Facial Expression Detection and Classification Based on Selective State Space

arXiv: https://arxiv.org/abs/2405.01828 · HTML: https://mlsanity.com/paper/2405.01828/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 3 May 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Speech and audio SSMs](https://mlsanity.com/ssm/speech-and-audio-ssms/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation](https://mlsanity.com/paper/2401.04722/) (2024)
- [Diagonal State Spaces are as Effective as Structured State Spaces](https://mlsanity.com/paper/2203.14343/) (2022)
- [VM-UNet: Vision Mamba UNet for Medical Image Segmentation](https://mlsanity.com/paper/2402.02491/) (2024)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining](https://mlsanity.com/paper/2402.03302/) (2024)
- [RSMamba: Remote Sensing Image Classification with State Space Model](https://mlsanity.com/paper/2403.19654/) (2024)
- [MedMamba: Vision Mamba for Medical Image Classification](https://mlsanity.com/paper/2403.03849/) (2024)
- [RS-Mamba for Large Remote Sensing Image Dense Prediction](https://mlsanity.com/paper/2404.02668/) (2024)
- [Vivim: a Video Vision Mamba for Medical Video Segmentation](https://mlsanity.com/paper/2401.14168/) (2024)
- [nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Model](https://mlsanity.com/paper/2402.03526/) (2024)
- [Res-VMamba: Fine-Grained Food Category Visual Classification Using Selective State Space Models with Deep Residual Learning](https://mlsanity.com/paper/2402.15761/) (2024)
- [Impact of facial landmark localization on facial expression recognition](https://mlsanity.com/paper/1905.10784/) (2019)
- [Facial Emotion Recognition using Convolutional Neural Networks](https://mlsanity.com/paper/1910.05602/) (2019)

## Built on by (most cited first, 7 of 7)

- [Mamba-MOC: A Multicategory Remote Object Counting via State Space Model](https://mlsanity.com/paper/2501.06697/) (2025)
- [MambaFlow: A Mamba-Centric Architecture for End-to-End Optical Flow Estimation](https://mlsanity.com/paper/2503.07046/) (2025)
- [Venturing into Uncharted Waters: The Navigation Compass from Transformer to Mamba](https://mlsanity.com/paper/2406.16722/) (2024)
- [Proxy Prompt: Endowing SAM and SAM 2 with Auto-Interactive-Prompt for Medical Segmentation](https://mlsanity.com/paper/2502.03501/) (2025)
- [UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation](https://mlsanity.com/paper/2505.15234/) (2025)
- [Facial Expression Recognition with YOLOv11 and YOLOv12: A Comparative Study](https://mlsanity.com/paper/2511.10940/) (2025)
- [Facial Expression Recognition in the Deep Learning Era: A Systematic Multi-Criteria Review of Methods, Models, Datasets, Performance, Challenges, and Future Research Directions](https://mlsanity.com/paper/2606.08612/) (2026)

## Abstract (arXiv)

Facial Expression Recognition (FER) plays a pivotal role in understanding human emotional cues. However, traditional FER methods based on visual information have some limitations, such as preprocessing, feature extraction, and multi-stage classification procedures. These not only increase computational complexity but also require a significant amount of computing resources. Considering Convolutional Neural Network (CNN)-based FER schemes frequently prove inadequate in identifying the deep, long-distance dependencies embedded within facial expression images, and the Transformer's inherent quadratic computational complexity, this paper presents the FER-YOLO-Mamba model, which integrates the principles of Mamba and YOLO technologies to facilitate efficient coordination in facial expression image recognition and localization. Within the FER-YOLO-Mamba model, we further devise a FER-YOLO-VSS dual-branch module, which combines the inherent strengths of convolutional layers in local feature extraction with the exceptional capability of State Space Models (SSMs) in revealing long-distance dependencies. To the best of our knowledge, this is the first Vision Mamba model designed for facial expression detection and classification. To evaluate the performance of the proposed FER-YOLO-Mamba model, we conducted experiments on two benchmark datasets, RAF-DB and SFEW. The experimental results indicate that the FER-YOLO-Mamba model achieved better results compared to other models. The code is available from https://github.com/SwjtuMa/FER-YOLO-Mamba.