# Decoupling recognition and transcription in Mandarin ASR

arXiv: https://arxiv.org/abs/2108.01129 · HTML: https://mlsanity.com/paper/2108.01129/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 53 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [Intermediate Loss Regularization for CTC-based Speech Recognition](https://mlsanity.com/paper/2102.03216/) (2021)
- [Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictions](https://mlsanity.com/paper/2104.02724/) (2021)
- [Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytes](https://mlsanity.com/paper/1811.09021/) (2018)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition](https://mlsanity.com/paper/2012.05481/) (2020)
- [Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition](https://mlsanity.com/paper/2104.01721/) (2021)
- [Applying Wav2vec2.0 to Speech Recognition in Various Low-resource Languages](https://mlsanity.com/paper/2012.12121/) (2020)
- [Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition](https://mlsanity.com/paper/2005.07903/) (2020)
- [Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input](https://mlsanity.com/paper/2010.15025/) (2020)
- [Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognition](https://mlsanity.com/paper/2005.04862/) (2020)
- [Simplified Self-Attention for Transformer-based End-to-End Speech Recognition](https://mlsanity.com/paper/2005.10463/) (2020)
- [Cascade RNN-Transducer: Syllable Based Streaming On-device Mandarin Speech Recognition with a Syllable-to-Character Converter](https://mlsanity.com/paper/2011.08469/) (2020)
- [Non-autoregressive Transformer-based End-to-end ASR using BERT](https://mlsanity.com/paper/2104.04805/) (2021)
- [Efficient conformer-based speech recognition with linear attention](https://mlsanity.com/paper/2104.06865/) (2021)
- [A Further Study of Unsupervised Pre-training for Transformer Based Speech Recognition](https://mlsanity.com/paper/2005.09862/) (2020)
- [Boundary and Context Aware Training for CIF-based Non-Autoregressive End-to-end ASR](https://mlsanity.com/paper/2104.04702/) (2021)
- [A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinese](https://mlsanity.com/paper/1805.06239/) (2018)
- [Darts-Conformer: Towards Efficient Gradient-Based Neural Architecture Search For End-to-End ASR](https://mlsanity.com/paper/2104.02868/) (2021)
- [WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition](https://mlsanity.com/paper/2104.03587/) (2021)
- [Gated Recurrent Fusion with Joint Training Framework for Robust End-to-End Speech Recognition](https://mlsanity.com/paper/2011.04249/) (2020)
- [Transformer with Bidirectional Decoder for Speech Recognition](https://mlsanity.com/paper/2008.04481/) (2020)
- [Decoupling Pronunciation and Language for End-to-end Code-switching Automatic Speech Recognition](https://mlsanity.com/paper/2010.14798/) (2020)
- [Multi-head Monotonic Chunkwise Attention For Online Speech Recognition](https://mlsanity.com/paper/2005.00205/) (2020)
- [Transformer-based Online Speech Recognition with Decoder-end Adaptive Computation Steps](https://mlsanity.com/paper/2011.13834/) (2020)
- [Adversarial Joint Training with Self-Attention Mechanism for Robust End-to-End Speech Recognition](https://mlsanity.com/paper/2104.01471/) (2021)

## Built on by (most cited first, 6 of 6)

- [Regeneration Learning: A Learning Paradigm for Data Generation](https://mlsanity.com/paper/2301.08846/) (2023)
- [Large Language Model Should Understand Pinyin for Chinese ASR Error Correction](https://mlsanity.com/paper/2409.13262/) (2024)
- [SCaLa: Supervised Contrastive Learning for End-to-End Speech Recognition](https://mlsanity.com/paper/2110.04187/) (2021)
- [Romanization Encoding For Multilingual ASR](https://mlsanity.com/paper/2407.04368/) (2024)
- [Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition](https://mlsanity.com/paper/2205.11998/) (2022)
- [Transducers with Pronunciation-aware Embeddings for Automatic Speech Recognition](https://mlsanity.com/paper/2404.04295/) (2024)

## Abstract (arXiv)

Much of the recent literature on automatic speech recognition (ASR) is taking an end-to-end approach. Unlike English where the writing system is closely related to sound, Chinese characters (Hanzi) represent meaning, not sound. We propose factoring audio -> Hanzi into two sub-tasks: (1) audio -> Pinyin and (2) Pinyin -> Hanzi, where Pinyin is a system of phonetic transcription of standard Chinese. Factoring the audio -> Hanzi task in this way achieves 3.9% CER (character error rate) on the Aishell-1 corpus, the best result reported on this dataset so far.