mlsanityData to 7 Oct 2026

Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyond

arXiv 2505.04621 ↗First posted 7 May 2025cs.SD cs.AI cs.LGcited by 5

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We introduce Audio-SDS, a generalization of Score Distillation Sampling (SDS) to text-conditioned audio diffusion models. While SDS was initially designed for text-to-3D generation using image diffusion, its core idea of distilling a powerful generative prior into a separate parametric representation extends to the audio domain. Leveraging a single pretrained model, Audio-SDS enables a broad range of tasks without requiring specialized datasets. In particular, we demonstrate how Audio-SDS can guide physically informed impact sound simulations, calibrate FM-synthesis parameters, and perform prompt-specified source separation. Our findings illustrate the versatility of distillation-based methods across modalities and establish a robust foundation for future work using generative priors in audio tasks.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 69 of its 97 references that are in the map. First posted on arXiv 7 May 2025.

Builds on

69 of 97 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Physics-Driven Diffusion Models for Impact Sound Synthesis from Videos29 Mar
  2. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI23 Mar
  3. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models30 Jan
  4. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan
  5. 2022Diffusion-based Generative Speech Source Separation31 Oct
  6. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 63

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  7. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  9. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  10. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  12. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
38 more
  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  2. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  3. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  4. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  5. 2017Population Based Training of Neural Networkscited by 351
  6. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  7. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  8. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  9. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  10. 2023Text-to-3D using Gaussian Splattingcited by 299
  11. 2022AudioGen: Textually Guided Audio Generationcited by 292
  12. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  13. 2018Adversarial Audio Synthesiscited by 222
  14. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  15. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
  16. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  17. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  18. 2020DDSP: Differentiable Digital Signal Processingcited by 177
  19. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  20. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  21. 2018Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separationcited by 125
  22. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  23. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 109
  24. 2019Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functionscited by 104
  25. 2019Music Source Separation in the Waveform Domaincited by 98
  26. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Modelscited by 91
  27. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  28. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  29. 2023Using Large Language Models for Hyperparameter Optimizationcited by 71
  30. 2022Learning Neural Acoustic Fieldscited by 65
  31. 2018Negative Momentum for Improved Game Dynamicscited by 59
  32. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  33. 2023Graph Metanetworks for Processing Diverse Neural Architecturescited by 55
  34. 2020Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesiscited by 54
  35. 2023Separate Anything You Describecited by 43
  36. 2024Score Distillation via Reparametrized DDIMcited by 41
  37. 2024LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesiscited by 39
  38. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 33

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Addressable Memory for Video World Modelscited by 11
  2. 2026Motion Attribution for Video Generationcited by 4
  3. 2025PromptReverb: Multimodal Room Impulse Response Generation Through Latent Rectified Flow Matchingcited by 0
  4. 2026Probability-Flow Distillation: Distribution Matching in Parameter Spacecited by 0
  5. 2026Objects as Audio-Visual Modal Sound Fieldscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementation
  2. 2026SwiftAudio: Data-Efficient Caption-Only Distillation for One-Step Text-to-Audio Diffusion-based Generation
  3. 2026Fast Text-to-Audio Generation with One-Step Sampling via Energy-Scoring and Auxiliary Contextual Representation Distillation ↗
  4. 2025DGMO: Training-Free Audio Source Separation through Diffusion-Guided Mask Optimization
  5. 2024Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models
  6. 2025High-Quality Sound Separation Across Diverse Categories via Visually-Guided Generative Modeling
  7. 2025Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis
  8. 2026AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation