# Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese

arXiv: https://arxiv.org/abs/1804.10752 · HTML: https://mlsanity.com/paper/1804.10752/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2018.

## Built on by (most cited first, 20 of 20)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [End-to-End Speaker-Attributed ASR with Transformer](https://mlsanity.com/paper/2104.02128/) (2021)
- [Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT](https://mlsanity.com/paper/2102.07594/) (2021)
- [Internal Language Model Estimation Through Explicit Context Vector Learning for Attention-based Encoder-decoder ASR](https://mlsanity.com/paper/2201.11627/) (2022)
- [LipFormer: Learning to Lipread Unseen Speakers based on Visual-Landmark Transformers](https://mlsanity.com/paper/2302.02141/) (2023)
- [Folding Attention: Memory and Power Optimization for On-Device Transformer-based Streaming Speech Recognition](https://mlsanity.com/paper/2309.07988/) (2023)
- [Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition](https://mlsanity.com/paper/2102.07259/) (2021)
- [Streaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolution](https://mlsanity.com/paper/2110.05241/) (2021)
- [Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios](https://mlsanity.com/paper/2104.02232/) (2021)
- [On the Effectiveness of Pinyin-Character Dual-Decoding for End-to-End Mandarin Chinese ASR](https://mlsanity.com/paper/2201.10792/) (2022)
- [PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models](https://mlsanity.com/paper/2406.07801/) (2024)
- [End-to-End Automatic Speech Recognition with Deep Mutual Learning](https://mlsanity.com/paper/2102.08154/) (2021)
- [Towards Language Modelling in the Speech Domain Using Sub-word Linguistic Units](https://mlsanity.com/paper/2111.00610/) (2021)
- [Semantically Corrected Amharic Automatic Speech Recognition](https://mlsanity.com/paper/2404.13362/) (2024)
- [Tricks and Plugins to GBM on Images and Sequences](https://mlsanity.com/paper/2203.00761/) (2022)
- [Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition](https://mlsanity.com/paper/2205.11998/) (2022)
- [Tricks and Plug-ins for Gradient Boosting with Transformers](https://mlsanity.com/paper/2508.02924/) (2025)
- [Multi-Level Embedding Conformer Framework for Bengali Automatic Speech Recognition](https://mlsanity.com/paper/2601.09710/) (2025)
- [ASSET: Adaptive Speech Segmentation via Reinforcement Learning](https://mlsanity.com/paper/2610.14309/) (2026)

## Abstract (arXiv)

Sequence-to-sequence attention-based models have recently shown very promising results on automatic speech recognition (ASR) tasks, which integrate an acoustic, pronunciation and language model into a single neural network. In these models, the Transformer, a new sequence-to-sequence attention-based model relying entirely on self-attention without using RNNs or convolutions, achieves a new single-model state-of-the-art BLEU on neural machine translation (NMT) tasks. Since the outstanding performance of the Transformer, we extend it to speech and concentrate on it as the basic architecture of sequence-to-sequence attention-based model on Mandarin Chinese ASR tasks. Furthermore, we investigate a comparison between syllable based model and context-independent phoneme (CI-phoneme) based model with the Transformer in Mandarin Chinese. Additionally, a greedy cascading decoder with the Transformer is proposed for mapping CI-phoneme sequences and syllable sequences into word sequences. Experiments on HKUST datasets demonstrate that syllable based model with the Transformer performs better than CI-phoneme based counterpart, and achieves a character error rate (CER) of 28.77%, which is competitive to the state-of-the-art CER of 28.0% by the joint CTC-attention based encoder-decoder network.