# Tiny Transducer: A Highly-efficient Speech Recognition Model on Edge Devices

arXiv: https://arxiv.org/abs/2101.06856 · HTML: https://mlsanity.com/paper/2101.06856/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2021.

## Built on by (most cited first, 19 of 19)

- [PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition](https://mlsanity.com/paper/2106.05933/) (2021)
- [Tied & Reduced RNN-T Decoder](https://mlsanity.com/paper/2109.07513/) (2021)
- [Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network](https://mlsanity.com/paper/2305.12493/) (2023)
- [Accelerating RNN-T Training and Inference Using CTC guidance](https://mlsanity.com/paper/2210.16481/) (2022)
- [TinyML: Tools, Applications, Challenges, and Future Research Directions](https://mlsanity.com/paper/2303.13569/) (2023)
- [Two Stage Contextual Word Filtering for Context bias in Unified Streaming and Non-streaming Transducer](https://mlsanity.com/paper/2301.06735/) (2023)
- [Adaptive Contextual Biasing for Transducer Based Streaming Speech Recognition](https://mlsanity.com/paper/2306.00804/) (2023)
- [FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularization](https://mlsanity.com/paper/2104.02882/) (2021)
- [Improving the fusion of acoustic and text representations in RNN-T](https://mlsanity.com/paper/2201.10240/) (2022)
- [Blank-regularized CTC for Frame Skipping in Neural Transducer](https://mlsanity.com/paper/2305.11558/) (2023)
- [Scribosermo: Fast Speech-to-Text models for German and other Languages](https://mlsanity.com/paper/2110.07982/) (2021)
- [Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing](https://mlsanity.com/paper/2211.01522/) (2022)
- [Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter](https://mlsanity.com/paper/2406.07096/) (2024)
- [Blank Collapse: Compressing CTC emission for the faster decoding](https://mlsanity.com/paper/2210.17017/) (2022)
- [Key Frame Mechanism For Efficient Conformer Based End-to-end Speech Recognition](https://mlsanity.com/paper/2310.14954/) (2023)
- [ASR Bundestag: A Large-Scale political debate dataset in German](https://mlsanity.com/paper/2302.06008/) (2023)
- [On the compression of shallow non-causal ASR models using knowledge distillation and tied-and-reduced decoder for low-latency on-device speech recognition](https://mlsanity.com/paper/2312.09842/) (2023)
- [End-to-end Speech Recognition with similar length speech and text](https://mlsanity.com/paper/2510.10453/) (2025)
- [Peeking Into The Future For Contextual Biasing](https://mlsanity.com/paper/2512.17657/) (2025)

## Abstract (arXiv)

This paper proposes an extremely lightweight phone-based transducer model with a tiny decoding graph on edge devices. First, a phone synchronous decoding (PSD) algorithm based on blank label skipping is first used to speed up the transducer decoding process. Then, to decrease the deletion errors introduced by the high blank score, a blank label deweighting approach is proposed. To reduce parameters and computation, deep feedforward sequential memory network (DFSMN) layers are used in the transducer encoder, and a CNN-based stateless predictor is adopted. SVD technology compresses the model further. WFST-based decoding graph takes the context-independent (CI) phone posteriors as input and allows us to flexibly bias user-specific information. Finally, with only 0.9M parameters after SVD, our system could give a relative 9.1% - 20.5% improvement compared with a bigger conventional hybrid system on edge devices.