# Improving Fast-slow Encoder based Transducer with Streaming Deliberation

arXiv: https://arxiv.org/abs/2212.07650 · HTML: https://mlsanity.com/paper/2212.07650/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2022.

## Built on by (most cited first, 5 of 5)

- [Prompting Large Language Models with Speech Recognition Abilities](https://mlsanity.com/paper/2307.11795/) (2023)
- [Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition](https://mlsanity.com/paper/2310.06434/) (2023)
- [Joint Beam Search Integrating CTC, Attention, and Transducer Decoders](https://mlsanity.com/paper/2406.02950/) (2024)
- [Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing](https://mlsanity.com/paper/2509.16622/) (2025)
- [Partial Rewriting for Multi-Stage ASR](https://mlsanity.com/paper/2312.09463/) (2023)

## Abstract (arXiv)

This paper introduces a fast-slow encoder based transducer with streaming deliberation for end-to-end automatic speech recognition. We aim to improve the recognition accuracy of the fast-slow encoder based transducer while keeping its latency low by integrating a streaming deliberation model. Specifically, the deliberation model leverages partial hypotheses from the streaming fast encoder and implicitly learns to correct recognition errors. We modify the parallel beam search algorithm for fast-slow encoder based transducer to be efficient and compatible with the deliberation model. In addition, the deliberation model is designed to process streaming data. To further improve the deliberation performance, a simple text augmentation approach is explored. We also compare LSTM and Conformer models for encoding partial hypotheses. Experiments on Librispeech and in-house data show relative WER reductions (WERRs) from 3% to 5% with a slight increase in model size and negligible extra token emission latency compared with fast-slow encoder based transducer. Compared with vanilla neural transducers, the proposed deliberation model together with fast-slow encoder based transducer obtains relative 10-11% WERRs on Librispeech and around relative 6% WERR on in-house data with smaller emission delays.