mlsanityData to 7 Oct 2026

Audio, speech and music

891 papers2 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: speech, audio, diffusion, generation, music

Key facts. Audio, speech and music: 891 papers on arXiv since 2020 (data through 7 Oct 2026). 188 of its 891 mapped papers (21%) cite “DiffWave: A Versatile Diffusion Model for Audio Synthesis” (2020, arXiv 2009.09761). 167 of its 891 mapped papers (19%) cite “AudioLDM: Text-to-Audio Generation with Latent Diffusion Models” (2023, arXiv 2301.12503). 20 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

20 new papers, 17 Sep – 7 Oct; 2 this week.

  1. This week · 2
  2. Real-Time Joint Audio-Video Generation by Parallel Adapter Composition
  3. Smorph: Playable Sound Morphing with Diffusion Models
  4. Earlier · 18
  5. Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation also fits: Video generation
  6. DriftTTS: Few-Step Text-to-Speech Without Distillation via Distribution-Matching Drift
  7. NEUROTOKEN: Joint Source and Directional AAD with Envelope Decoding via Conditional Flow Matching
  8. MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion
  9. ECHO-G: Embodied Co-speech Humanoid mOtion Generation
  10. Harmonizing Spectral Evolution in Conditional Flow Matching for TTS
  11. JoyAI-Voice 2.0: A Full-Continuous Autoregressive Speech Generation Model with Semantic-Acoustic Joint Representation
  12. Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL also fits: Diffusion and flow policies for robotics
All 20
  1. Distill Locally, Schedule Globally: Flow Maps for Few-Step Text-to-Speech
  2. SEmoEdit: Probing and Harnessing the Editability of Pre-trained Speech Flows
  3. NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speech
  4. Depth through recurrence: Looped transformers for flow-matching TTS
  5. Off-manifold robustness in synthesizer inversion with joint distribution flow matching
  6. ReaFlow-TTS: Realization-Conditioned Flow Matching for High-Quality and Controllable Speech Synthesis
  7. UNITE-AUDIO: Joint Learning of Continuous Tokenization and Latent Flow Matching for Text-to-Audio Generation
  8. P2Flow: Phoneme-aware Progressive Flow Matching for Extreme Speech Super-Resolution
  9. Generative Learning for Ambisonic Upscaling
  10. DiffVQE2: An Efficient Low-delay Diffusion Model for Acoustic Echo and Noise Control

Foundations

Most cited by papers here

  1. DiffWave: A Versatile Diffusion Model for Audio Synthesis2020cited by 188 here
  2. AudioLDM: Text-to-Audio Generation with Latent Diffusion Models2023cited by 167 here
  3. AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining2023cited by 128 here
  4. Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech2021cited by 118 here
  5. Speech Enhancement and Dereverberation with Diffusion-based Generative Models2022cited by 102 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2020Denoising Diffusion Probabilistic Modelscited by 516 here
  2. 2020Denoising Diffusion Implicit Modelscited by 242 here
  3. 2021Improved Denoising Diffusion Probabilistic Modelscited by 89 here
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 191 here
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 324 here
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 93 here
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 89 here
  8. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 167 here
  9. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 91 here
  10. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 128 here
14 more it builds on, not in this chain
  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 188 here
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 269 here
  3. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 118 here
  4. 2022Conditional Diffusion Probabilistic Model for Speech Enhancementcited by 78 here
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 76 here
  6. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 77 here
  7. 2022Classifier-Free Diffusion Guidancecited by 210 here
  8. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Modelscited by 102 here
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 106 here
  10. 2022Flow Matching for Generative Modelingcited by 185 here
  11. 2022Scalable Diffusion Models with Transformerscited by 194 here
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 73 here
  13. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 76 here
  14. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 99 here

The 24 most cited of 1,562.

Browse

30 of 891, ranked 6–35

  1. 6F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching202499
  2. 7Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models202391
  3. 8Conditional Diffusion Probabilistic Model for Speech Enhancement202278
  4. 9Diffsound: Discrete Diffusion Model for Text-to-sound Generation202277
  5. 10NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers202369
  6. 11Matcha-TTS: A fast TTS architecture with conditional flow matching202361
  7. 12Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain202261
  8. 13StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation202259
  9. 14EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions202456
  10. 15Diff-TTS: A Denoising Diffusion Model for Text-to-Speech202156
All 30
  1. 16NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models202450
  2. 17Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model202348
  3. 18ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech202247
  4. 19Noise2Music: Text-conditioned Music Generation with Diffusion Models202345
  5. 20CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models202444
  6. 21FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis202242
  7. 22MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies202338
  8. 23Symbolic Music Generation with Diffusion Models202138
  9. 24A Study on Speech Enhancement Based on Diffusion Probabilistic Model202138
  10. 25Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme202138
  11. 26Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation202337
  12. 27MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation202237
  13. 28Long-form music generation with latent diffusion202435
  14. 29AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models202335
  15. 30Audiobox: Unified Audio Generation with Natural Language Prompts202335
  16. 31Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency202434
  17. 32Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer202434
  18. 33Mousai: Text-to-Music Generation with Long-Context Latent Diffusion202334
  19. 34Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generation202333
  20. 35Solving Audio Inverse Problems with a Diffusion Model202233