# ZeroPrompt: Streaming Acoustic Encoders are Zero-Shot Masked LMs

arXiv: https://arxiv.org/abs/2305.10649 · HTML: https://mlsanity.com/paper/2305.10649/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Generative Pre-Training for Speech with Autoregressive Predictive Coding](https://mlsanity.com/paper/1910.12607/) (2019)
- [FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization](https://mlsanity.com/paper/2010.11148/) (2020)
- [WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2203.15455/) (2022)
- [Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition](https://mlsanity.com/paper/2012.05481/) (2020)
- [Streaming Transformer-based Acoustic Models Using Self-attention with Augmented Memory](https://mlsanity.com/paper/2005.08042/) (2020)
- [Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping](https://mlsanity.com/paper/1902.06450/) (2019)
- [Dissecting User-Perceived Latency of On-Device E2E Speech Recognition](https://mlsanity.com/paper/2104.02207/) (2021)
- [CUSIDE: Chunking, Simulating Future Context and Decoding for Streaming ASR](https://mlsanity.com/paper/2203.16758/) (2022)
- [Adaptable End-to-End ASR Models using Replaceable Internal LMs and Residual Softmax](https://mlsanity.com/paper/2302.08579/) (2023)
- [TrimTail: Low-Latency Streaming ASR with Simple but Effective Spectrogram-Level Length Penalty](https://mlsanity.com/paper/2211.00522/) (2022)
- [Peak-First CTC: Reducing the Peak Latency of CTC Models by Applying Peak-First Regularization](https://mlsanity.com/paper/2211.03284/) (2022)
- [Improving Streaming End-to-End ASR on Transformer-based Causal Models with Encoder States Revision Strategies](https://mlsanity.com/paper/2207.02495/) (2022)

## Built on by (most cited first, 13 of 13)

- [Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review](https://mlsanity.com/paper/2402.10350/) (2024)
- [Exploring Diverse Methods in Visual Question Answering](https://mlsanity.com/paper/2404.13565/) (2024)
- [Accelerating Parallel Sampling of Diffusion Models](https://mlsanity.com/paper/2402.09970/) (2024)
- [Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approach](https://mlsanity.com/paper/2408.06634/) (2024)
- [Ensemble Methodology:Innovations in Credit Default Prediction Using LightGBM, XGBoost, and LocalEnsemble](https://mlsanity.com/paper/2402.17979/) (2024)
- [Integration of Mamba and Transformer -- MAT for Long-Short Range Time Series Forecasting with Application to Weather Dynamics](https://mlsanity.com/paper/2409.08530/) (2024)
- [Research on Credit Risk Early Warning Model of Commercial Banks Based on Neural Network Algorithm](https://mlsanity.com/paper/2405.10762/) (2024)
- [Leveraging Contrastive Learning and Self-Training for Multimodal Emotion Recognition with Limited Labeled Samples](https://mlsanity.com/paper/2409.04447/) (2024)
- [Optimization and Application of Cloud-based Deep Learning Architecture for Multi-Source Data Prediction](https://mlsanity.com/paper/2410.12642/) (2024)
- [MEDFuse: Multimodal EHR Data Fusion with Masked Lab-Test Modeling and Large Language Models](https://mlsanity.com/paper/2407.12309/) (2024)
- [Enhancing Exchange Rate Forecasting with Explainable Deep Learning Models](https://mlsanity.com/paper/2410.19241/) (2024)
- [Wormhole: Concept-Aware Deep Representation Learning for Co-Evolving Sequences](https://mlsanity.com/paper/2409.13857/) (2024)
- [SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2603.10005/) (2026)

## Abstract (arXiv)

In this paper, we present ZeroPrompt (Figure 1-(a)) and the corresponding Prompt-and-Refine strategy (Figure 3), two simple but effective training-free methods to decrease the Token Display Time (TDT) of streaming ASR models without any accuracy loss. The core idea of ZeroPrompt is to append zeroed content to each chunk during inference, which acts like a prompt to encourage the model to predict future tokens even before they were spoken. We argue that streaming acoustic encoders naturally have the modeling ability of Masked Language Models and our experiments demonstrate that ZeroPrompt is engineering cheap and can be applied to streaming acoustic encoders on any dataset without any accuracy loss. Specifically, compared with our baseline models, we achieve 350 ~ 700ms reduction on First Token Display Time (TDT-F) and 100 ~ 400ms reduction on Last Token Display Time (TDT-L), with theoretically and experimentally equal WER on both Aishell-1 and Librispeech datasets.