# Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios

arXiv: https://arxiv.org/abs/2104.02232 · HTML: https://mlsanity.com/paper/2104.02232/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [A Comparative Study on Transformer vs RNN in Speech Applications](https://mlsanity.com/paper/1909.06317/) (2019)
- [Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition](https://mlsanity.com/paper/2010.10759/) (2020)
- [A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency](https://mlsanity.com/paper/2003.12710/) (2020)
- [Transformer-based Acoustic Modeling for Hybrid Speech Recognition](https://mlsanity.com/paper/1910.09799/) (2019)
- [Transformer-Transducer: End-to-End Speech Recognition with Self-Attention](https://mlsanity.com/paper/1910.12977/) (2019)
- [FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization](https://mlsanity.com/paper/2010.11148/) (2020)
- [Cascaded encoders for unifying streaming and non-streaming ASR](https://mlsanity.com/paper/2010.14606/) (2020)
- [Towards Fast and Accurate Streaming End-to-End ASR](https://mlsanity.com/paper/2004.11544/) (2020)
- [Alignment Restricted Streaming Recurrent Neural Network Transducer](https://mlsanity.com/paper/2011.03072/) (2020)
- [Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model](https://mlsanity.com/paper/1712.01541/) (2017)
- [Efficient minimum word error rate training of RNN-Transducer for end-to-end speech recognition](https://mlsanity.com/paper/2007.13802/) (2020)
- [Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping](https://mlsanity.com/paper/1902.06450/) (2019)
- [Optimizing Speech Recognition For The Edge](https://mlsanity.com/paper/1909.12408/) (2019)
- [Self-Attentional Acoustic Models](https://mlsanity.com/paper/1803.09519/) (2018)
- [Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition](https://mlsanity.com/paper/1507.06947/) (2015)
- [Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages](https://mlsanity.com/paper/1806.05059/) (2018)
- [Transformer Transducer: One Model Unifying Streaming and Non-streaming Speech Recognition](https://mlsanity.com/paper/2010.03192/) (2020)
- [Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese](https://mlsanity.com/paper/1804.10752/) (2018)
- [Low Latency End-to-End Streaming Speech Recognition with a Scout Network](https://mlsanity.com/paper/2003.10369/) (2020)
- [Attention based on-device streaming speech recognition with large speech corpus](https://mlsanity.com/paper/2001.00577/) (2020)
- [Scaling Up Online Speech Recognition Using ConvNets](https://mlsanity.com/paper/2001.09727/) (2020)
- [Transformer in action: a comparative study of transformer-based acoustic models for large scale speech recognition applications](https://mlsanity.com/paper/2010.14665/) (2020)
- [Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces](https://mlsanity.com/paper/2005.09150/) (2020)
- [Memory-efficient Speech Recognition on Smart Devices](https://mlsanity.com/paper/2102.11531/) (2021)
- [Weak-Attention Suppression For Transformer Based Speech Recognition](https://mlsanity.com/paper/2005.09137/) (2020)
- [Analyzing the Quality and Stability of a Streaming End-to-End On-Device Speech Recognizer](https://mlsanity.com/paper/2006.01416/) (2020)
- [Streaming Attention-Based Models with Augmented Memory for End-to-End Speech Recognition](https://mlsanity.com/paper/2011.07120/) (2020)

## Built on by (most cited first, 3 of 3)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Learning a Dual-Mode Speech Recognition Model via Self-Pruning](https://mlsanity.com/paper/2207.11906/) (2022)
- [Comparison and Analysis of New Curriculum Criteria for End-to-End ASR](https://mlsanity.com/paper/2208.05782/) (2022)

## Abstract (arXiv)

Often, the storage and computational constraints of embeddeddevices demand that a single on-device ASR model serve multiple use-cases / domains. In this paper, we propose aFlexibleTransducer(FlexiT) for on-device automatic speech recognition to flexibly deal with multiple use-cases / domains with different accuracy and latency requirements. Specifically, using a single compact model, FlexiT provides a fast response for voice commands, and accurate transcription but with more latency for dictation. In order to achieve flexible and better accuracy and latency trade-offs, the following techniques are used. Firstly, we propose using domain-specific altering of segment size for Emformer encoder that enables FlexiT to achieve flexible de-coding. Secondly, we use Alignment Restricted RNNT loss to achieve flexible fine-grained control on token emission latency for different domains. Finally, we add a domain indicator vector as an additional input to the FlexiT model. Using the combination of techniques, we show that a single model can be used to improve WERs and real time factor for dictation scenarios while maintaining optimal latency for voice commands use-cases