mlsanityData to 7 Oct 2026

Lip reading using external viseme decoding

arXiv 2104.04784 ↗First posted 10 Apr 2021cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Lip-reading is the operation of recognizing speech from lip movements. This is a difficult task because the movements of the lips when pronouncing the words are similar for some of them. Viseme is used to describe lip movements during a conversation. This paper aims to show how to use external text data (for viseme-to-character mapping) by dividing video-to-character into two stages, namely converting video to viseme, and then converting viseme to character by using separate models. Our proposed method improves word error rate by 4% compared to the normal sequence to sequence lip-reading model on the BBC-Oxford Lip Reading Sentences 2 (LRS2) dataset.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 10 of its 24 references that are in the map. First posted on arXiv 10 Apr 2021.

Builds on

10 of 24 references.

References · 10

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Listen, Attend and Spellcited by 542
  3. 2015Attention-Based Models for Speech Recognitioncited by 400
  4. 2018LRS3-TED: a large-scale dataset for visual speech recognitioncited by 301
  5. 2016Lip Reading Sentences in the Wildcited by 226
  6. 2020Lipreading using Temporal Convolutional Networkscited by 98
  7. 2019ASR is all you need: cross-modal distillation for lip readingcited by 85
  8. 2018Deep Lip Reading: a comparison of models and an online applicationcited by 26
  9. 2017Decoding visemes: improving machine lipreadingcited by 7
  10. 2020Synchronous Bidirectional Learning for Multilingual Lip Readingcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 0; 2025: 1.

5 papers.

  1. 2023A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Datasetcited by 4
  2. 2023Word-level Persian Lipreading Datasetcited by 2
  3. 2023Leveraging Visemes for Better Visual Speech Representation and Lip Readingcited by 2
  4. 2023Improving the Gap in Visual Speech Recognition Between Normal and Silent Speech Based on Metric Learningcited by 1
  5. 2025Learning Speaker-Invariant Visual Features for Lipreadingcited by 0