DeepASMR: LLM-Based Zero-Shot ASMR Speech Generation for Anyone of Any Voice
Not on the map. Why?
Abstract (arXiv)
While modern Text-to-Speech (TTS) systems achieve high fidelity for read-style speech, they struggle to generate Autonomous Sensory Meridian Response (ASMR), a specialized, low-intensity speech style essential for relaxation. The inherent challenges include ASMR's subtle, often unvoiced characteristics and the demand for zero-shot speaker adaptation. In this paper, we introduce DeepASMR, the first framework designed for zero-shot ASMR generation. We demonstrate that a single short snippet of a speaker's ordinary, read-style speech is sufficient to synthesize high-fidelity ASMR in their voice, eliminating the need for whispered training data from the target speaker. Methodologically, we first identify that discrete speech tokens provide a soft factorization of ASMR style from speaker timbre. Leveraging this insight, we propose a two-stage pipeline incorporating a Large Language Model (LLM) for content-style encoding and a flow-matching acoustic decoder for timbre reconstruction. Furthermore, we contribute DeepASMR-DB, a comprehensive 670-hour English-Chinese multi-speaker ASMR speech corpus, and introduce a novel evaluation protocol integrating objective metrics, human listening tests, LLM-based scoring and unvoiced speech analysis. Extensive experiments confirm that DeepASMR achieves state-of-the-art naturalness and style fidelity in ASMR generation for anyone of any voice, while maintaining competitive performance on normal speech synthesis.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 62 references that are in the map. First posted on arXiv 22 Jan 2026.
Builds on
39 of 62 references.
References · 39
- 2017Attention Is All You Needcited by 59,932
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
- 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
- 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
27 more
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
- 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
- 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
- 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 169
- 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
- 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 149
- 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
- 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
- 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 109
- 2022Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkitcited by 92
- 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 85
- 2024Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Modelcited by 70
- 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 69
- 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 44
- 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 33
- 2023Boosting Large Language Model for Speech Synthesis: An Empirical Studycited by 21
- 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
- 2024StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matchingcited by 14
- 2025CoVoMix2: Advancing Zero-Shot Dialogue Generation with Fully Non-Autoregressive Flow Matchingcited by 11
- 2025FlexiCodec: A Dynamic Neural Audio Codec for Low Frame Ratescited by 9
- 2019Voice Conversion for Whispered Speech Synthesiscited by 7
- 2021Whispered and Lombard Neural Speech Synthesiscited by 6
- 2023WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech Interactionscited by 6
- 2025Advanced Zero-Shot Text-to-Speech for Background Removal and Preservation with Controllable Masked Speech Predictioncited by 5
- 2025Discl-VC: Disentangled Discrete Tokens and In-Context Learning for Controllable Zero-Shot Voice Conversioncited by 5
- 2023Vocoder-Free Non-Parallel Conversion of Whispered Speech With Masked Cycle-Consistent Generative Adversarial Networkscited by 2
- 2024Improvement Speaker Similarity for Zero-Shot Any-to-Any Voice Conversion of Whispered and Regular Speechcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
- 2024FlashSpeech: Efficient Zero-Shot Speech Synthesis
- 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers
- 2025Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching
- 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models
- 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias
- 2024Takin-VC: Expressive Zero-Shot Voice Conversion via Adaptive Hybrid Content Encoding and Enhanced Timbre Modeling
- 2025InstructAudio: Unified speech and music generation with natural language instruction