Personalized Korean Lipreading as Visual Speech Recognition: Transfer, Census and Adaptation on OLKAVS
Not on the map. Why?
Abstract (arXiv)
We present a personalized Korean visual speech recognition (VSR) system and quantify, on the nine-camera OLKAVS corpus, the gap between the population-level benchmark score and an individual user's error. A video-only Conformer initialized from English-trained weights attains 9.95 - 12.19% character error rate (CER) under the corpus protocol against the published 26.64, and 19.00 - 21.52 on unseen wording. Per speaker, CER spans 1.0 to 52.2%, with seen wording lowering CER by 7.0 - 9.0 points and professional delivery and spontaneous speech raising it by 8.5 - 10.5 and 12.7 points. A low-rank adapter with 4.6% of the parameters, trained on 4 to 29 minutes of the user's frontal video, lowers the CER of twelve high-error speakers by 2.13 to 3.58 points, transfers to every camera without loss, and keeps 85% of the full fine-tuning gain at 12% of its cost to other speakers. Cameras above the mouth plane add about six CER points as a constant offset that training on all views keeps small.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 24 Sep 2026.
Builds on
15 of 29 references.
References · 15
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
- 2021End-to-end Audio-visual Speech Recognition with Conformerscited by 151
- 2023Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labelscited by 114
- 2023SynthVSR: Scaling Up Visual Speech Recognition With Synthetic Supervisioncited by 21
- 2022Speaker-adaptive Lip Reading with User-dependent Paddingcited by 17
- 2019Towards Pose-invariant Lip-Readingcited by 12
- 2025VALLR: Visual ASR Language Model for Lip Readingcited by 11
- 2025Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognitioncited by 9
- 2017End-to-End Multi-View Lipreadingcited by 7
- 2018Visual-Only Recognition of Normal, Whispered and Silent Speechcited by 7
Built on by
Not cited yet.