# Towards Lipreading Sentences with Active Appearance Models

arXiv: https://arxiv.org/abs/1805.11688 · HTML: https://mlsanity.com/paper/1805.11688/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2018.

## Built on by (most cited first, 4 of 4)

- [LipSound2: Self-Supervised Pre-Training for Lip-to-Speech Reconstruction and Lip Reading](https://mlsanity.com/paper/2112.04748/) (2021)
- [LRWR: Large-Scale Benchmark for Lip Reading in Russian language](https://mlsanity.com/paper/2109.06692/) (2021)
- [Is Lip Region-of-Interest Sufficient for Lipreading?](https://mlsanity.com/paper/2205.14295/) (2022)
- [Target Speaker Lipreading by Audio-Visual Self-Distillation Pretraining and Speaker Adaptation](https://mlsanity.com/paper/2502.05758/) (2025)

## Abstract (arXiv)

Automatic lipreading has major potential impact for speech recognition, supplementing and complementing the acoustic modality. Most attempts at lipreading have been performed on small vocabulary tasks, due to a shortfall of appropriate audio-visual datasets. In this work we use the publicly available TCD-TIMIT database, designed for large vocabulary continuous audio-visual speech recognition. We compare the viseme recognition performance of the most widely used features for lipreading, Discrete Cosine Transform (DCT) and Active Appearance Models (AAM), in a traditional Hidden Markov Model (HMM) framework. We also exploit recent advances in AAM fitting. We found the DCT to outperform AAM by more than 6% for a viseme recognition task with 56 speakers. The overall accuracy of the DCT is quite low (32-34%). We conclude that a fundamental rethink of the modelling of visual features may be needed for this task.