# MMME: A Spontaneous Multi-Modal Micro-Expression Dataset Enabling Visual-Physiological Fusion

arXiv: https://arxiv.org/abs/2506.09834 · HTML: https://mlsanity.com/paper/2506.09834/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 121 references that are in the map.
- First posted on arXiv 11 Jun 2025.

## Builds on (most cited first, 18 of 18)

- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution](https://mlsanity.com/paper/1608.01041/) (2016)
- [Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms](https://mlsanity.com/paper/2201.12728/) (2022)
- [Deep Learning for Micro-expression Recognition: A Survey](https://mlsanity.com/paper/2107.02823/) (2021)
- [Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition](https://mlsanity.com/paper/1902.03634/) (2019)
- [Less is More: Micro-expression Recognition from Video using Apex Frame](https://mlsanity.com/paper/1606.01721/) (2016)
- [Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition](https://mlsanity.com/paper/2006.09674/) (2020)
- [Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition](https://mlsanity.com/paper/1805.08417/) (2018)
- [Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions](https://mlsanity.com/paper/1901.04656/) (2019)
- [Feature refinement: An expression-specific feature learning and fusion method for micro-expression recognition](https://mlsanity.com/paper/2101.04838/) (2021)
- [HTNet for micro-expression recognition](https://mlsanity.com/paper/2307.14637/) (2023)
- [Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition](https://mlsanity.com/paper/2112.05851/) (2021)
- [Micro-expression detection in long videos using optical flow and recurrent neural networks](https://mlsanity.com/paper/1903.10765/) (2019)
- [An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge](https://mlsanity.com/paper/2012.11307/) (2020)
- [Micro-expression spotting: A new benchmark](https://mlsanity.com/paper/2007.12421/) (2020)
- [DFME: A New Benchmark for Dynamic Facial Micro-expression Recognition](https://mlsanity.com/paper/2301.00985/) (2023)
- [Could Micro-Expressions be Quantified? Electromyography Gives Affirmative Evidence](https://mlsanity.com/paper/2409.00017/) (2024)
- [Micro-Expression Recognition by Motion Feature Extraction based on Pre-training](https://mlsanity.com/paper/2407.07345/) (2024)

## Abstract (arXiv)

Micro-expressions (MEs) are subtle, fleeting nonverbal cues that reveal an individual's genuine emotional state. Their analysis has attracted considerable interest due to its promising applications in fields such as healthcare, criminal investigation, and human-computer interaction. However, existing ME research is limited to single visual modality, overlooking the rich emotional information conveyed by other physiological modalities, resulting in ME recognition and spotting performance far below practical application needs. Therefore, exploring the cross-modal association mechanism between ME visual features and physiological signals (PS), and developing a multimodal fusion framework, represents a pivotal step toward advancing ME analysis. This study introduces a novel ME dataset, MMME, which, for the first time, enables synchronized collection of facial action signals (MEs), central nervous system signals (EEG), and peripheral PS (PPG, RSP, SKT, EDA, and ECG). By overcoming the constraints of existing ME corpora, MMME comprises 634 MEs, 2,841 macro-expressions (MaEs), and 2,890 trials of synchronized multimodal PS, establishing a robust foundation for investigating ME neural mechanisms and conducting multimodal fusion-based analyses. Extensive experiments validate the dataset's reliability and provide benchmarks for ME analysis, demonstrating that integrating MEs with PS significantly enhances recognition and spotting performance. To the best of our knowledge, MMME is the most comprehensive ME dataset to date in terms of modality diversity. It provides critical data support for exploring the neural mechanisms of MEs and uncovering the visual-physiological synergistic effects, driving a paradigm shift in ME research from single-modality visual analysis to multimodal fusion. The dataset will be publicly available upon acceptance of this paper.