# Kaggle Competition: Cantonese Audio-Visual Speech Recognition for In-car Commands

arXiv: https://arxiv.org/abs/2207.02663 · HTML: https://mlsanity.com/paper/2207.02663/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 10 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 10 of 10)

- [Deep Audio-Visual Speech Recognition](https://mlsanity.com/paper/1809.02108/) (2018)
- [End-to-end Audio-visual Speech Recognition with Conformers](https://mlsanity.com/paper/2102.06657/) (2021)
- [Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation](https://mlsanity.com/paper/2203.06386/) (2022)
- [ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation](https://mlsanity.com/paper/2112.06223/) (2021)
- [Multimodal End-to-End Sparse Model for Emotion Recognition](https://mlsanity.com/paper/2103.09666/) (2021)
- [Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization](https://mlsanity.com/paper/2109.02401/) (2021)
- [Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition](https://mlsanity.com/paper/2009.09629/) (2020)
- [Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection](https://mlsanity.com/paper/2004.13432/) (2020)
- [Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset](https://mlsanity.com/paper/2201.02419/) (2022)
- [CI-AVSR: A Cantonese Audio-Visual Speech Dataset for In-car Command Recognition](https://mlsanity.com/paper/2201.03804/) (2022)

## Abstract (arXiv)

With the rise of deep learning and intelligent vehicles, the smart assistant has become an essential in-car component to facilitate driving and provide extra functionalities. In-car smart assistants should be able to process general as well as car-related commands and perform corresponding actions, which eases driving and improves safety. However, in this research field, most datasets are in major languages, such as English and Chinese. There is a huge data scarcity issue for low-resource languages, hindering the development of research and applications for broader communities. Therefore, it is crucial to have more benchmarks to raise awareness and motivate the research in low-resource languages. To mitigate this problem, we collect a new dataset, namely Cantonese In-car Audio-Visual Speech Recognition (CI-AVSR), for in-car speech recognition in the Cantonese language with video and audio data. Together with it, we propose Cantonese Audio-Visual Speech Recognition for In-car Commands as a new challenge for the community to tackle low-resource speech recognition under in-car scenarios.