# LRS3-TED: a large-scale dataset for visual speech recognition

arXiv: https://arxiv.org/abs/1809.00496 · HTML: https://mlsanity.com/paper/1809.00496/

## Key facts

- Cited by 308 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2018.

## Built on by (most cited first, 30 of 308)

- [Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction](https://mlsanity.com/paper/2201.02184/) (2022)
- [End-to-end Audio-visual Speech Recognition with Conformers](https://mlsanity.com/paper/2102.06657/) (2021)
- [GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis](https://mlsanity.com/paper/2301.13430/) (2023)
- [DF40: Toward Next-Generation Deepfake Detection](https://mlsanity.com/paper/2406.13495/) (2024)
- [Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels](https://mlsanity.com/paper/2303.14307/) (2023)
- [Identity-Preserving Talking Face Generation with Landmark and Appearance Priors](https://mlsanity.com/paper/2305.08293/) (2023)
- [Emotional Speech-Driven Animation with Content-Emotion Disentanglement](https://mlsanity.com/paper/2306.08990/) (2023)
- [Synchformer: Efficient Synchronization from Sparse Cues](https://mlsanity.com/paper/2401.16423/) (2024)
- [Visual Speech Recognition for Multiple Languages in the Wild](https://mlsanity.com/paper/2202.13084/) (2022)
- [Self-Supervised Video Forensics by Audio-Visual Anomaly Detection](https://mlsanity.com/paper/2301.01767/) (2023)
- [SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis](https://mlsanity.com/paper/2404.04104/) (2024)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Robust Self-Supervised Audio-Visual Speech Recognition](https://mlsanity.com/paper/2201.01763/) (2022)
- [SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces](https://mlsanity.com/paper/2306.10799/) (2023)
- [Responsive Listening Head Generation: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2112.13548/) (2021)
- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [Sub-word Level Lip Reading With Visual Attention](https://mlsanity.com/paper/2110.07603/) (2021)
- [AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection](https://mlsanity.com/paper/2406.02951/) (2024)
- [Imitating Arbitrary Talking Style for Realistic Audio-DrivenTalking Face Synthesis](https://mlsanity.com/paper/2111.00203/) (2021)
- [Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos](https://mlsanity.com/paper/2207.11094/) (2022)
- [Jointly Learning Visual and Auditory Speech Representations from Raw Data](https://mlsanity.com/paper/2212.06246/) (2022)
- [LiRA: Learning Visual Speech Representations from Audio through Self-supervision](https://mlsanity.com/paper/2106.09171/) (2021)
- [Learning in Audio-visual Context: A Review, Analysis, and New Perspective](https://mlsanity.com/paper/2208.09579/) (2022)
- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Watch or Listen: Robust Audio-Visual Speech Recognition with Visual Corruption Modeling and Reliability Scoring](https://mlsanity.com/paper/2303.08536/) (2023)
- [GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation](https://mlsanity.com/paper/2305.00787/) (2023)
- [Large Language Models are Strong Audio-Visual Speech Recognition Learners](https://mlsanity.com/paper/2409.12319/) (2024)
- [Audio-Visual Synchronisation in the wild](https://mlsanity.com/paper/2112.04432/) (2021)
- [u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modality](https://mlsanity.com/paper/2207.07036/) (2022)

## Abstract (arXiv)

This paper introduces a new multi-modal dataset for visual and audio-visual speech recognition. It includes face tracks from over 400 hours of TED and TEDx videos, along with the corresponding subtitles and word alignment boundaries. The new dataset is substantially larger in scale compared to other public datasets that are available for general research.