# Emotion recognition in talking-face videos using persistent entropy and neural networks

arXiv: https://arxiv.org/abs/2110.13571 · HTML: https://mlsanity.com/paper/2110.13571/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Survey on Emotional Body Gesture Recognition](https://mlsanity.com/paper/1801.07481/) (2018)
- [Persistent Homology of Complex Networks for Dynamic State Detection](https://mlsanity.com/paper/1904.07403/) (2019)
- [A persistent homology approach to heart rate variability analysis with an application to sleep-wake classification](https://mlsanity.com/paper/1908.06856/) (2019)
- [On the Bootstrap for Persistence Diagrams and Landscapes](https://mlsanity.com/paper/1311.0376/) (2013)
- [Probabilistic Fr'echet Means for Time Varying Persistence Diagrams](https://mlsanity.com/paper/1307.6530/) (2013)
- [Towards Emotion Recognition: A Persistent Entropy Application](https://mlsanity.com/paper/1811.09607/) (2018)
- [Towards personalized diagnosis of Glioblastoma in Fluid-attenuated inversion recovery (FLAIR) by topological interpretable machine learning](https://mlsanity.com/paper/1912.08167/) (2019)
- [Audio-video Emotion Recognition in the Wild using Deep Hybrid Networks](https://mlsanity.com/paper/2002.09023/) (2020)
- [Automatic Recognition of Facial Displays of Unfelt Emotions](https://mlsanity.com/paper/1707.04061/) (2017)

## Built on by (most cited first, 1 of 1)

- [Topological Data Analysis Applications in Natural Language Processing: A Survey](https://mlsanity.com/paper/2411.10298/) (2024)

## Abstract (arXiv)

The automatic recognition of a person's emotional state has become a very active research field that involves scientists specialized in different areas such as artificial intelligence, computer vision or psychology, among others. Our main objective in this work is to develop a novel approach, using persistent entropy and neural networks as main tools, to recognise and classify emotions from talking-face videos. Specifically, we combine audio-signal and image-sequence information to compute a topology signature(a 9-dimensional vector) for each video. We prove that small changes in the video produce small changes in the signature. These topological signatures are used to feed a neural network to distinguish between the following emotions: neutral, calm, happy, sad, angry, fearful, disgust, and surprised. The results reached are promising and competitive, beating the performance reached in other state-of-the-art works found in the literature.