# Pronunciation-aware unique character encoding for RNN Transducer-based Mandarin speech recognition

arXiv: https://arxiv.org/abs/2207.14578 · HTML: https://mlsanity.com/paper/2207.14578/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 29 Jul 2022.

## Builds on (most cited first, 22 of 22)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency](https://mlsanity.com/paper/2003.12710/) (2020)
- [End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results](https://mlsanity.com/paper/1412.1602/) (2014)
- [Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability](https://mlsanity.com/paper/2007.15188/) (2020)
- [Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducer](https://mlsanity.com/paper/2010.13878/) (2020)
- [Simplified Self-Attention for Transformer-based End-to-End Speech Recognition](https://mlsanity.com/paper/2005.10463/) (2020)
- [On Addressing Practical Challenges for RNN-Transducer](https://mlsanity.com/paper/2105.00858/) (2021)
- [Cascade RNN-Transducer: Syllable Based Streaming On-device Mandarin Speech Recognition with a Syllable-to-Character Converter](https://mlsanity.com/paper/2011.08469/) (2020)
- [Building competitive direct acoustics-to-word models for English conversational speech recognition](https://mlsanity.com/paper/1712.03133/) (2017)
- [Language model fusion for streaming end to end speech recognition](https://mlsanity.com/paper/2104.04487/) (2021)
- [Sub-Character Tokenization for Chinese Pretrained Language Models](https://mlsanity.com/paper/2106.00400/) (2021)
- [A comparable study of modeling units for end-to-end Mandarin speech recognition](https://mlsanity.com/paper/1805.03832/) (2018)
- [Subword and Crossword Units for CTC Acoustic Models](https://mlsanity.com/paper/1712.06855/) (2017)

## Built on by (most cited first, 1 of 1)

- [Transducers with Pronunciation-aware Embeddings for Automatic Speech Recognition](https://mlsanity.com/paper/2404.04295/) (2024)

## Abstract (arXiv)

For Mandarin end-to-end (E2E) automatic speech recognition (ASR) tasks, compared to character-based modeling units, pronunciation-based modeling units could improve the sharing of modeling units in model training but meet homophone problems. In this study, we propose to use a novel pronunciation-aware unique character encoding for building E2E RNN-T-based Mandarin ASR systems. The proposed encoding is a combination of pronunciation-base syllable and character index (CI). By introducing the CI, the RNN-T model can overcome the homophone problem while utilizing the pronunciation information for extracting modeling units. With the proposed encoding, the model outputs can be converted into the final recognition result through a one-to-one mapping. We conducted experiments on Aishell and MagicData datasets, and the experimental results showed the effectiveness of the proposed method.