mlsanityData to 7 Oct 2026

DeepASMR: LLM-Based Zero-Shot ASMR Speech Generation for Anyone of Any Voice

arXiv 2601.15596 ↗First posted 22 Jan 2026cs.SD cs.AI eess.AScited by 1

Not on the map. Why?

Abstract (arXiv)

While modern Text-to-Speech (TTS) systems achieve high fidelity for read-style speech, they struggle to generate Autonomous Sensory Meridian Response (ASMR), a specialized, low-intensity speech style essential for relaxation. The inherent challenges include ASMR's subtle, often unvoiced characteristics and the demand for zero-shot speaker adaptation. In this paper, we introduce DeepASMR, the first framework designed for zero-shot ASMR generation. We demonstrate that a single short snippet of a speaker's ordinary, read-style speech is sufficient to synthesize high-fidelity ASMR in their voice, eliminating the need for whispered training data from the target speaker. Methodologically, we first identify that discrete speech tokens provide a soft factorization of ASMR style from speaker timbre. Leveraging this insight, we propose a two-stage pipeline incorporating a Large Language Model (LLM) for content-style encoding and a flow-matching acoustic decoder for timbre reconstruction. Furthermore, we contribute DeepASMR-DB, a comprehensive 670-hour English-Chinese multi-speaker ASMR speech corpus, and introduce a novel evaluation protocol integrating objective metrics, human listening tests, LLM-based scoring and unvoiced speech analysis. Extensive experiments confirm that DeepASMR achieves state-of-the-art naturalness and style fidelity in ASMR generation for anyone of any voice, while maintaining competitive performance on normal speech synthesis.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 62 references that are in the map. First posted on arXiv 22 Jan 2026.

Builds on

39 of 62 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  7. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  8. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  9. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  10. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  11. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  12. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
27 more
  1. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  2. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  3. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
  4. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 169
  5. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  6. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 149
  7. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
  8. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  9. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 109
  10. 2022Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkitcited by 92
  11. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 85
  12. 2024Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Modelcited by 70
  13. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 69
  14. 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 44
  15. 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 33
  16. 2023Boosting Large Language Model for Speech Synthesis: An Empirical Studycited by 21
  17. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  18. 2024StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matchingcited by 14
  19. 2025CoVoMix2: Advancing Zero-Shot Dialogue Generation with Fully Non-Autoregressive Flow Matchingcited by 11
  20. 2025FlexiCodec: A Dynamic Neural Audio Codec for Low Frame Ratescited by 9
  21. 2019Voice Conversion for Whispered Speech Synthesiscited by 7
  22. 2021Whispered and Lombard Neural Speech Synthesiscited by 6
  23. 2023WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech Interactionscited by 6
  24. 2025Advanced Zero-Shot Text-to-Speech for Background Removal and Preservation with Controllable Masked Speech Predictioncited by 5
  25. 2025Discl-VC: Disentangled Discrete Tokens and In-Context Learning for Controllable Zero-Shot Voice Conversioncited by 5
  26. 2023Vocoder-Free Non-Parallel Conversion of Whispered Speech With Masked Cycle-Consistent Generative Adversarial Networkscited by 2
  27. 2024Improvement Speaker Similarity for Zero-Shot Any-to-Any Voice Conversion of Whispered and Regular Speechcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
  2. 2024FlashSpeech: Efficient Zero-Shot Speech Synthesis
  3. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers
  4. 2025Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching
  5. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models
  6. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias
  7. 2024Takin-VC: Expressive Zero-Shot Voice Conversion via Adaptive Hybrid Content Encoding and Enhanced Timbre Modeling
  8. 2025InstructAudio: Unified speech and music generation with natural language instruction