# Analysis of Visual Features for Continuous Lipreading in Spanish

arXiv: https://arxiv.org/abs/2311.12468 · HTML: https://mlsanity.com/paper/2311.12468/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 33 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 4 of 4)

- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [Towards Estimating the Upper Bound of Visual-Speech Recognition: The Visual Lip-Reading Feasibility Database](https://mlsanity.com/paper/1704.08028/) (2017)
- [Lip Reading Using Convolutional Auto Encoders as Feature Extractor](https://mlsanity.com/paper/1805.12371/) (2018)

## Built on by (most cited first, 1 of 1)

- [Comparison of Conventional Hybrid and CTC/Attention Decoders for Continuous Visual Speech Recognition](https://mlsanity.com/paper/2402.13004/) (2024)

## Abstract (arXiv)

During a conversation, our brain is responsible for combining information obtained from multiple senses in order to improve our ability to understand the message we are perceiving. Different studies have shown the importance of presenting visual information in these situations. Nevertheless, lipreading is a complex task whose objective is to interpret speech when audio is not available. By dispensing with a sense as crucial as hearing, it will be necessary to be aware of the challenge that this lack presents. In this paper, we propose an analysis of different speech visual features with the intention of identifying which of them is the best approach to capture the nature of lip movements for natural Spanish and, in this way, dealing with the automatic visual speech recognition task. In order to estimate our system, we present an audiovisual corpus compiled from a subset of the RTVE database, which has been used in the Albayz'in evaluations. We employ a traditional system based on Hidden Markov Models with Gaussian Mixture Models. Results show that, although the task is difficult, in restricted conditions we obtain recognition results which determine that using eigenlips in combination with deep features is the best visual approach.