# CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition Challenge

arXiv: https://arxiv.org/abs/2406.10313 · HTML: https://mlsanity.com/paper/2406.10313/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 14 Jun 2024.

## Builds on (most cited first, 18 of 18)

- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Deep Audio-Visual Speech Recognition](https://mlsanity.com/paper/1809.02108/) (2018)
- [RetinaFace: Single-stage Dense Face Localisation in the Wild](https://mlsanity.com/paper/1905.00641/) (2019)
- [End-to-end Audio-visual Speech Recognition with Conformers](https://mlsanity.com/paper/2102.06657/) (2021)
- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels](https://mlsanity.com/paper/2303.14307/) (2023)
- [Intermediate Loss Regularization for CTC-based Speech Recognition](https://mlsanity.com/paper/2102.03216/) (2021)
- [Visual Speech Recognition for Multiple Languages in the Wild](https://mlsanity.com/paper/2202.13084/) (2022)
- [E-Branchformer: Branchformer with Enhanced merging for speech recognition](https://mlsanity.com/paper/2210.00077/) (2022)
- [Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding](https://mlsanity.com/paper/2207.02971/) (2022)
- [LipNet: End-to-End Sentence-level Lipreading](https://mlsanity.com/paper/1611.01599/) (2016)
- [Large-Scale Visual Speech Recognition](https://mlsanity.com/paper/1807.05162/) (2018)
- [LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild](https://mlsanity.com/paper/1810.06990/) (2018)
- [WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2203.15455/) (2022)
- [Lip-reading with Densely Connected Temporal Convolutional Networks](https://mlsanity.com/paper/2009.14233/) (2020)

## Built on by (most cited first, 5 of 5)

- [AlignVSR: Audio-Visual Cross-Modal Alignment for Visual Speech Recognition](https://mlsanity.com/paper/2410.16438/) (2024)
- [CNVSRC 2024: The Second Chinese Continuous Visual Speech Recognition Challenge](https://mlsanity.com/paper/2506.02010/) (2025)
- [Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing](https://mlsanity.com/paper/2506.02012/) (2025)
- [Quantitative Analysis of Audio-Visual Tasks: An Information-Theoretic Perspective](https://mlsanity.com/paper/2409.19575/) (2024)
- [VALLR-Pin: Uncertainty-Factorized Visual Speech Recognition for Mandarin with Pinyin Guidance](https://mlsanity.com/paper/2512.20032/) (2025)

## Abstract (arXiv)

The first Chinese Continuous Visual Speech Recognition Challenge aimed to probe the performance of Large Vocabulary Continuous Visual Speech Recognition (LVC-VSR) on two tasks: (1) Single-speaker VSR for a particular speaker and (2) Multi-speaker VSR for a set of registered speakers. The challenge yielded highly successful results, with the best submission significantly outperforming the baseline, particularly in the single-speaker task. This paper comprehensively reviews the challenge, encompassing the data profile, task specifications, and baseline system construction. It also summarises the representative techniques employed by the submitted systems, highlighting the most effective approaches. Additional information and resources about this challenge can be accessed through the official website at http://cnceleb.org/competition.