# CNVSRC 2024: The Second Chinese Continuous Visual Speech Recognition Challenge

arXiv: https://arxiv.org/abs/2506.02010 · HTML: https://mlsanity.com/paper/2506.02010/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 20 of 20)

- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction](https://mlsanity.com/paper/2201.02184/) (2022)
- [Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels](https://mlsanity.com/paper/2303.14307/) (2023)
- [E-Branchformer: Branchformer with Enhanced merging for speech recognition](https://mlsanity.com/paper/2210.00077/) (2022)
- [Combining Residual Networks with LSTMs for Lipreading](https://mlsanity.com/paper/1703.04105/) (2017)
- [Large-Scale Visual Speech Recognition](https://mlsanity.com/paper/1807.05162/) (2018)
- [MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verification](https://mlsanity.com/paper/2203.15249/) (2022)
- [Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Reading](https://mlsanity.com/paper/2204.01725/) (2022)
- [Transformer-Based Video Front-Ends for Audio-Visual Speech Recognition for Single and Multi-Person Video](https://mlsanity.com/paper/2201.10439/) (2022)
- [U2++: Unified Two-pass Bidirectional End-to-end Model for Speech Recognition](https://mlsanity.com/paper/2106.05642/) (2021)
- [Training Strategies for Improved Lip-reading](https://mlsanity.com/paper/2209.01383/) (2022)
- [Structured State Space Decoder for Speech Recognition and Synthesis](https://mlsanity.com/paper/2210.17098/) (2022)
- [Multi-Temporal Lip-Audio Memory for Visual Speech Recognition](https://mlsanity.com/paper/2305.04542/) (2023)
- [CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition Challenge](https://mlsanity.com/paper/2406.10313/) (2024)
- [Visual Speech Recognition](https://mlsanity.com/paper/1409.1411/) (2014)
- [Enhancing Lip Reading with Multi-Scale Video and Multi-Encoder](https://mlsanity.com/paper/2404.05466/) (2024)
- [The NPU-ASLP-LiAuto System Description for Visual Speech Recognition in CNVSRC 2023](https://mlsanity.com/paper/2401.06788/) (2024)

## Built on by (most cited first, 1 of 1)

- [VALLR-Pin: Uncertainty-Factorized Visual Speech Recognition for Mandarin with Pinyin Guidance](https://mlsanity.com/paper/2512.20032/) (2025)

## Abstract (arXiv)

This paper presents the second Chinese Continuous Visual Speech Recognition Challenge (CNVSRC 2024), which builds on CNVSRC 2023 to advance research in Chinese Large Vocabulary Continuous Visual Speech Recognition (LVC-VSR). The challenge evaluates two test scenarios: reading in recording studios and Internet speech. CNVSRC 2024 uses the same datasets as its predecessor CNVSRC 2023, which involves CN-CVS for training and CNVSRC-Single/Multi for development and evaluation. However, CNVSRC 2024 introduced two key improvements: (1) a stronger baseline system, and (2) an additional dataset, CN-CVS2-P1, for open tracks to improve data volume and diversity. The new challenge has demonstrated several important innovations in data preprocessing, feature extraction, model design, and training strategies, further pushing the state-of-the-art in Chinese LVC-VSR. More details and resources are available at the official website.