mlsanityData to 14 Oct 2026

Personalized Korean Lipreading as Visual Speech Recognition: Transfer, Census and Adaptation on OLKAVS

arXiv 2609.28988 ↗First posted 24 Sep 2026eess.AS cs.CL cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a personalized Korean visual speech recognition (VSR) system and quantify, on the nine-camera OLKAVS corpus, the gap between the population-level benchmark score and an individual user's error. A video-only Conformer initialized from English-trained weights attains 9.95 - 12.19% character error rate (CER) under the corpus protocol against the published 26.64, and 19.00 - 21.52 on unseen wording. Per speaker, CER spans 1.0 to 52.2%, with seen wording lowering CER by 7.0 - 9.0 points and professional delivery and spontaneous speech raising it by 8.5 - 10.5 and 12.7 points. A low-rank adapter with 4.6% of the parameters, trained on 4 to 29 minutes of the user's frontal video, lowers the CER of twelve high-error speakers by 2.13 to 3.58 points, transfers to every camera without loss, and keeps 85% of the full fine-tuning gain at 12% of its cost to other speakers. Cameras above the mouth plane add about six CER points as a constant offset that training on all views keeps small.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

15 of 29 references.

References · 15

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  4. 2021End-to-end Audio-visual Speech Recognition with Conformerscited by 151
  5. 2023Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labelscited by 114
  6. 2023SynthVSR: Scaling Up Visual Speech Recognition With Synthetic Supervisioncited by 21
  7. 2022Speaker-adaptive Lip Reading with User-dependent Paddingcited by 17
  8. 2019Towards Pose-invariant Lip-Readingcited by 12
  9. 2025VALLR: Visual ASR Language Model for Lip Readingcited by 11
  10. 2025Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognitioncited by 9
  11. 2017End-to-End Multi-View Lipreadingcited by 7
  12. 2018Visual-Only Recognition of Normal, Whispered and Silent Speechcited by 7
3 more
  1. 2024Personalized Lip Reading: Adapting to Your Unique Lip Movements with Vision and Languagecited by 5
  2. 2023OLKAVS: An Open Large-Scale Korean Audio-Visual Speech Datasetcited by 3
  3. 2026The Lipreading Gap: Do VSR Models Perceive Visual Speech Like Human Lipreaders?cited by 3

Built on by

Not cited yet.