# LIP-RTVE: An Audiovisual Database for Continuous Spanish in the Wild

arXiv: https://arxiv.org/abs/2311.12457 · HTML: https://mlsanity.com/paper/2311.12457/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 54 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 9 of 9)

- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Deep Audio-Visual Speech Recognition](https://mlsanity.com/paper/1809.02108/) (2018)
- [LRS3-TED: a large-scale dataset for visual speech recognition](https://mlsanity.com/paper/1809.00496/) (2018)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [Visual Speech Recognition for Multiple Languages in the Wild](https://mlsanity.com/paper/2202.13084/) (2022)
- [Can We Read Speech Beyond the Lips? Rethinking RoI Selection for Deep Visual Speech Recognition](https://mlsanity.com/paper/2003.03206/) (2020)
- [Towards Estimating the Upper Bound of Visual-Speech Recognition: The Visual Lip-Reading Feasibility Database](https://mlsanity.com/paper/1704.08028/) (2017)
- [Comparing phonemes and visemes with DNN-based lipreading](https://mlsanity.com/paper/1805.02924/) (2018)

## Built on by (most cited first, 7 of 7)

- [Personalized Lip Reading: Adapting to Your Unique Lip Movements with Vision and Language](https://mlsanity.com/paper/2409.00986/) (2024)
- [Tailored Design of Audio-Visual Speech Recognition Models using Branchformers](https://mlsanity.com/paper/2407.06606/) (2024)
- [AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologies](https://mlsanity.com/paper/2402.13152/) (2024)
- [Comparison of Conventional Hybrid and CTC/Attention Decoders for Continuous Visual Speech Recognition](https://mlsanity.com/paper/2402.13004/) (2024)
- [Evaluation of End-to-End Continuous Spanish Lipreading in Different Data Conditions](https://mlsanity.com/paper/2502.00464/) (2025)
- [Bootstrapping Audiovisual Speech Recognition in Zero-AV-Resource Scenarios](https://mlsanity.com/paper/2603.08249/) (2026)
- [Improving Audiovisual Speech Recognition through Synthetic Visual Data Augmentation](https://mlsanity.com/paper/2609.31961/) (2026)

## Abstract (arXiv)

Speech is considered as a multi-modal process where hearing and vision are two fundamentals pillars. In fact, several studies have demonstrated that the robustness of Automatic Speech Recognition systems can be improved when audio and visual cues are combined to represent the nature of speech. In addition, Visual Speech Recognition, an open research problem whose purpose is to interpret speech by reading the lips of the speaker, has been a focus of interest in the last decades. Nevertheless, in order to estimate these systems in the currently Deep Learning era, large-scale databases are required. On the other hand, while most of these databases are dedicated to English, other languages lack sufficient resources. Thus, this paper presents a semi-automatically annotated audiovisual database to deal with unconstrained natural Spanish, providing 13 hours of data extracted from Spanish television. Furthermore, baseline results for both speaker-dependent and speaker-independent scenarios are reported using Hidden Markov Models, a traditional paradigm that has been widely used in the field of Speech Technologies.