# A Pilot Study on Mandarin Chinese Cued Speech

arXiv: https://arxiv.org/abs/2001.00731 · HTML: https://mlsanity.com/paper/2001.00731/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2020.

## Built on by (most cited first, 9 of 9)

- [A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insights](https://mlsanity.com/paper/2407.08428/) (2024)
- [Cross-Modal Mutual Learning for Cued Speech Recognition](https://mlsanity.com/paper/2212.01083/) (2022)
- [A Novel Interpretable and Generalizable Re-synchronization Model for Cued Speech based on a Multi-Cuer Corpus](https://mlsanity.com/paper/2306.02596/) (2023)
- [A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation](https://mlsanity.com/paper/2308.08849/) (2023)
- [Computation and Parameter Efficient Multi-Modal Fusion Transformer for Cued Speech Recognition](https://mlsanity.com/paper/2401.17604/) (2024)
- [Bridge to Non-Barrier Communication: Gloss-Prompted Fine-grained Cued Speech Gesture Generation with Diffusion Model](https://mlsanity.com/paper/2404.19277/) (2024)
- [MAVD: The First Open Large-Scale Mandarin Audio-Visual Dataset with Depth Information](https://mlsanity.com/paper/2306.02263/) (2023)
- [UniCUE: Unified Recognition and Generation Framework for Chinese Cued Speech Video-to-Speech Generation](https://mlsanity.com/paper/2506.04134/) (2025)
- [Cued-Agent: A Collaborative Multi-Agent System for Automatic Cued Speech Recognition](https://mlsanity.com/paper/2508.00391/) (2025)

## Abstract (arXiv)

Cued Speech (CS) is a communication system developed for deaf people, which exploits hand cues to complement speechreading at the phonetic level. Currently, it is estimated that CS has been adapted to over 60 languages; however, no official CS system is available for Mandarin Chinese. This article proposes a novel and efficient Mandarin Chinese CS system, satisfying the main criterion that the hand coding constitutes a complement to the lips movements. We propose to code vowels [i, u, y] as semiconsonants when they are followed by other Mandarin finals, which reduces the number of Mandarin finals to be coded from 36 to 16. We establish a coherent similarity between Mandarin Chinese and French vowels for the remaining 16 vowels, which allows us to take advantage of the French CS system. Furthermore, by investigating the lips viseme distribution based on a new corpus, an optimal allocation of the 16 Mandarin vowels to different hand positions is obtained. A Gaussian classifier was used to evaluate the average separability of different allocated vowel groups, which gives 92.08%, 92.33%, and 92.73% for the three speakers, respectively. The consonants are mainly designed according to their similarities with the French CS system, as well as some considerations on the special Mandarin consonants. In our system, the tones of Mandarin are coded with head movements.