# Multimodal Latent Emotion Recognition from Micro-expression and Physiological Signals

arXiv: https://arxiv.org/abs/2308.12156 · HTML: https://mlsanity.com/paper/2308.12156/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 44 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 11 of 11)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [OFF-ApexNet on Micro-expression Recognition System](https://mlsanity.com/paper/1805.08699/) (2018)
- [Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition](https://mlsanity.com/paper/1902.03634/) (2019)
- [Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods](https://mlsanity.com/paper/1511.00423/) (2015)
- [Less is More: Micro-expression Recognition from Video using Apex Frame](https://mlsanity.com/paper/1606.01721/) (2016)
- [Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition](https://mlsanity.com/paper/1805.08417/) (2018)
- [Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions](https://mlsanity.com/paper/1901.04656/) (2019)
- [Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition](https://mlsanity.com/paper/2112.05851/) (2021)
- [Revisiting Self-Supervised Contrastive Learning for Facial Expression Recognition](https://mlsanity.com/paper/2210.03853/) (2022)
- [A Novel Apex-Time Network for Cross-Dataset Micro-Expression Recognition](https://mlsanity.com/paper/1904.03699/) (2019)
- [Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition](https://mlsanity.com/paper/2210.05246/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)

## Abstract (arXiv)

This paper discusses the benefits of incorporating multimodal data for improving latent emotion recognition accuracy, focusing on micro-expression (ME) and physiological signals (PS). The proposed approach presents a novel multimodal learning framework that combines ME and PS, including a 1D separable and mixable depthwise inception network, a standardised normal distribution weighted feature fusion method, and depth/physiology guided attention modules for multimodal learning. Experimental results show that the proposed approach outperforms the benchmark method, with the weighted fusion method and guided attention modules both contributing to enhanced performance.