mlsanityData to 14 Oct 2026

Turning Whisper into Real-Time Transcription System

arXiv 2307.14743 ↗First posted 27 Jul 2023cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Whisper is one of the recent state-of-the-art multilingual speech recognition and translation models, however, it is not designed for real time transcription. In this paper, we build on top of Whisper and create Whisper-Streaming, an implementation of real-time speech transcription and translation of Whisper-like models. Whisper-Streaming uses local agreement policy with self-adaptive latency to enable streaming transcription. We show that Whisper-Streaming achieves high quality and 3.3 seconds latency on unsegmented long-form speech transcription test set, and we demonstrate its robustness and practical usability as a component in live transcription service at a multilingual conference.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 6 of its 12 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

6 of 12 references.

References · 6

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  3. 2020Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selectioncited by 49
  4. 2021Learning When to Translate for Streaming Speechcited by 30
  5. 2022CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022cited by 30
  6. 2021Lost in Interpreting: Speech Translation from Source or Interpreter?cited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 12; 2026: 7.

24 papers.

Sort:

By area:

  1. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 27
  2. 2024Building a Taiwanese Mandarin Spoken Language Model: A First Attemptcited by 23
  3. 2025SHANKS: Simultaneous Hearing and Thinking for Spoken Language Modelscited by 19
  4. 2025Simultaneous Translation with Offline Speech and LLM Models in CUNI Submission to IWSLT 2025cited by 8
  5. 2024Simul-Whisper: Attention-Guided Streaming Whisper with Truncation Detectioncited by 7
  6. 2025LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximationcited by 7
  7. 2025Can Vision-Language Models Answer Face to Face Questions in the Real-World?cited by 7
  8. 2026Voxtral Realtimecited by 6
  9. 2025WhisperKit: On-device Real-time ASR with Billion-Scale Transformerscited by 4
  10. 2023Long-form Simultaneous Speech Translation: Thesis Proposalcited by 3
  11. 2026From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Modelscited by 3
  12. 2024Assessing Latency in ASR Systems: A Methodological Perspective for Real-Time Usecited by 2
12 more
  1. 2025Quantization for OpenAI's Whisper Models: A Comparative Analysiscited by 2
  2. 2025ConsumerBench: Benchmarking Generative AI Applications on End-User Devicescited by 2
  3. 2025MLLP-VRAIN UPV system for the IWSLT 2025 Simultaneous Speech Translation Translation taskcited by 2
  4. 2025Scalable Offline ASR for Command-Style Dictation in Courtroomscited by 1
  5. 2026MeetBench-XL: Calibrated Multi-Dimensional Evaluation and Learned Dual-Policy Agents for Real-Time Meetingscited by 1
  6. 2024Whispy: Adapting STT Whisper Models to Real-Time Environmentscited by 0
  7. 2025LINC: Supporting Language Independent Communication and Comprehension to Enhance Contribution in Multilingual Collaborative Meetingscited by 0
  8. 2025WhisperRT -- Turning Whisper into a Causal Streaming Modelcited by 0
  9. 2026WhisperPipe: A Resource-Efficient Streaming Architecture for Real-Time Automatic Speech Recognitioncited by 0
  10. 2026Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agentscited by 0
  11. 2026X2Streaming-ASR: wait when uncertain, emit when ready for streaming ASRcited by 0
  12. 2026RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Executioncited by 0