# VIBEVOICE-ASR Technical Report

arXiv: https://arxiv.org/abs/2601.18184 · HTML: https://mlsanity.com/paper/2601.18184/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 21 references that are in the map.
- First posted on arXiv 26 Jan 2026.

## Builds on (most cited first, 17 of 17)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [pyannote.audio: neural building blocks for speaker diarization](https://mlsanity.com/paper/1911.01255/) (2019)
- [AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario](https://mlsanity.com/paper/2104.03603/) (2021)
- [Efficient Sequence Transduction by Jointly Predicting Tokens and Durations](https://mlsanity.com/paper/2304.06795/) (2023)
- [Multimodal Latent Language Modeling with Next-Token Diffusion](https://mlsanity.com/paper/2412.08635/) (2024)
- [VibeVoice Technical Report](https://mlsanity.com/paper/2508.19205/) (2025)
- [AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data](https://mlsanity.com/paper/2309.13905/) (2023)
- [SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Models](https://mlsanity.com/paper/2508.06372/) (2025)
- [Summary on The Multilingual Conversational Speech Language Model Challenge: Datasets, Tasks, Baselines, and Methods](https://mlsanity.com/paper/2509.13785/) (2025)
- [Train Short, Infer Long: Speech-LLM Enables Zero-Shot Streamable Joint ASR and Diarization on Long Audio](https://mlsanity.com/paper/2511.16046/) (2025)
- [Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control](https://mlsanity.com/paper/2601.03973/) (2026)
- [TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Grounding](https://mlsanity.com/paper/2601.06896/) (2026)

## Built on by (most cited first, 16 of 16)

- [Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders](https://mlsanity.com/paper/2606.07473/) (2026)
- [A Unified and Reproducible Experimentation Framework for Speech Understanding](https://mlsanity.com/paper/2605.30899/) (2026)
- [Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis](https://mlsanity.com/paper/2606.22811/) (2026)
- [SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models](https://mlsanity.com/paper/2607.05365/) (2026)
- [How to Recognize New Words: A Comparison Between Context Biasing Methods and Speech LLMs](https://mlsanity.com/paper/2608.05759/) (2026)
- [G-STAR: End-to-End Global Speaker-Tracking Attributed Recognition](https://mlsanity.com/paper/2603.10468/) (2026)
- [Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics](https://mlsanity.com/paper/2603.22709/) (2026)
- [GigaChat Audio: Time-aware Large Audio Language Model](https://mlsanity.com/paper/2607.10387/) (2026)
- [VibeVoice-ASR-BitNet Technical Report](https://mlsanity.com/paper/2607.21075/) (2026)
- [Leading-Silence Augmentation and Multi-Stage Synthetic Supervision for the Second MLC-SLM Challenge](https://mlsanity.com/paper/2608.14150/) (2026)
- [HearInContext: A Benchmark for Implicit Context in Speech Recognition](https://mlsanity.com/paper/2609.18680/) (2026)
- [Transsion's Speaker-Attributed Multilingual ASR System for the MLC-SLM 2026 Challenge](https://mlsanity.com/paper/2609.20833/) (2026)
- [The Bairong System for MLC-SLM 2026: Dynamic Question-Aware Evidence Routing for Multilingual Conversational Speech Understanding](https://mlsanity.com/paper/2609.22214/) (2026)
- [Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetings](https://mlsanity.com/paper/2609.39344/) (2026)
- [PERSIST: Who-What-When Memory Across Sessions for Full-Duplex Spoken Dialogue](https://mlsanity.com/paper/2610.07725/) (2026)
- [HINTT Submission to the 2nd MLC-SLM Challenge: Comparing Cascaded and Unified Approaches to Diarization and ASR](https://mlsanity.com/paper/2610.08063/) (2026)

## Abstract (arXiv)

This report presents VibeVoice-ASR, a general-purpose speech understanding framework built upon VibeVoice, designed to address the persistent challenges of context fragmentation and multi-speaker complexity in long-form audio (e.g., meetings, podcasts) that remain despite recent advancements in short-form speech recognition. Unlike traditional pipelined approaches that rely on audio chunking, VibeVoice-ASRsupports single-pass processing for up to 60 minutes of audio. It unifies Automatic Speech Recognition, Speaker Diarization, and Timestamping into a single end-to-end generation task. In addition, VibeVoice-ASR supports over 50 languages, requires no explicit language setting, and natively handles code-switching within and across utterances. Furthermore, we introduce a prompt-based context injection mechanism that allows users to supply customized conetxt, significantly improving accuracy on domain-specific terminology and polyphonic character disambiguation.