mlsanityData to 7 Oct 2026

Fast Timing-Conditioned Latent Audio Diffusion

arXiv 2402.04825 ↗First posted 7 Feb 2024cs.SD cs.LG eess.AScited by 178

Not on the map. Why?

Cited by 80 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generating long-form 44.1kHz stereo audio from text prompts can be computationally demanding. Further, most previous works do not tackle that music and sound effects naturally vary in their duration. Our research focuses on the efficient generation of long-form, variable-length stereo music and sounds at 44.1kHz using text prompts with a generative model. Stable Audio is based on latent diffusion, with its latent defined by a fully-convolutional variational autoencoder. It is conditioned on text prompts as well as timing embeddings, allowing for fine control over both the content and length of the generated music and sounds. Stable Audio is capable of rendering stereo signals of up to 95 sec at 44.1kHz in 8 sec on an A100 GPU. Despite its compute efficiency and fast inference, it is one of the best in two public text-to-music and -audio benchmarks and, differently from state-of-the-art models, can generate music with structure and stereo sounds.

Key facts. Cited by 178 papers in the map (data through 7 Oct 2026). Builds on 49 of its 60 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

49 of 60 references.

References · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  10. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  11. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  12. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
37 more
  1. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  2. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  3. 2022High Fidelity Neural Audio Compressioncited by 673
  4. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  5. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  6. 2020Jukebox: A Generative Model for Musiccited by 535
  7. 2023Simple and Controllable Music Generationcited by 480
  8. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  9. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  10. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 323
  11. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  12. 2018Adversarial Audio Synthesiscited by 222
  13. 2021Efficient Training of Audio Transformers with Patchoutcited by 222
  14. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 221
  15. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  16. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
  17. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Modelcited by 145
  18. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  19. 2023Audiobox: Unified Audio Generation with Natural Language Promptscited by 120
  20. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 97
  21. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 85
  22. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  23. 2022Solving Audio Inverse Problems with a Diffusion Modelcited by 62
  24. 2023VampNet: Music Generation via Masked Acoustic Token Modelingcited by 57
  25. 2023SingSong: Generating musical accompaniments from singingcited by 54
  26. 2023Efficient Neural Music Generationcited by 50
  27. 2024Masked Audio Generation using a Single Non-Autoregressive Transformercited by 50
  28. 2022Multi-instrument Music Synthesis with Spectrogram Diffusioncited by 42
  29. 2023JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Modelscited by 41
  30. 2021CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesiscited by 37
  31. 2022Full-band General Audio Synthesis with Score-based Diffusioncited by 32
  32. 2023StemGen: A music generation model that listenscited by 32
  33. 2023Controllable Music Production with Diffusion Models and Guidance Gradientscited by 29
  34. 2020Automatic multitrack mixing with a differentiable mixing console of neural audio effectscited by 27
  35. 2022Musika! Fast Infinite Waveform Music Generationcited by 24
  36. 2023CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Modelscited by 20
  37. 2023JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 65; 2025: 67; 2026: 46.

178 papers.

Sort:

By area:

  1. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  2. 2024Stable Audio Opencited by 192
  3. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 140
  4. 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 91
  5. 2024Long-form music generation with latent diffusioncited by 88
  6. 2024TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimizationcited by 67
  7. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  8. 2024Video-Guided Foley Sound Generation with Multimodal Controlscited by 49
  9. 2024Scaling Transformers for Low-Bitrate High-Quality Speech Codingcited by 48
  10. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 46
  11. 2025DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusioncited by 46
  12. 2024Linear Convergence of Diffusion Models Under the Manifold Hypothesiscited by 42
38 more
  1. 2024CleanDIFT: Diffusion Features without Noisecited by 42
  2. 2025Kling-Foley: Multimodal Diffusion Transformer for High-Quality Video-to-Audio Generationcited by 38
  3. 2024Foundation Models for Music: A Surveycited by 34
  4. 2024Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Modelscited by 33
  5. 2025ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and Editingcited by 33
  6. 2024EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformercited by 30
  7. 2024Taming Data and Transformers for Audio Generationcited by 28
  8. 2024DITTO-2: Distilled Diffusion Inference-Time T-Optimization for Music Generationcited by 26
  9. 2024SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Modelscited by 23
  10. 2025LeVo: High-Quality Song Generation with Multi-Preference Alignmentcited by 23
  11. 2024TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodingscited by 22
  12. 2024AV-Link: Temporally-Aligned Diffusion Features for Cross-Modal Audio-Video Generationcited by 21
  13. 2025FreeAudio: Training-Free Timing Planning for Controllable Long-Form Text-to-Audio Generationcited by 21
  14. 2024Images that Sound: Composing Images and Sounds on a Single Canvascited by 17
  15. 2024SimpleSpeech 2: Towards Simple and Efficient Text-to-Speech with Flow-based Scalar Latent Transformer Diffusion Modelscited by 17
  16. 2024ETTA: Elucidating the Design Space of Text-to-Audio Modelscited by 17
  17. 2025OmniAudio: Generating Spatial Audio from 360-Degree Videocited by 17
  18. 2024Presto! Distilling Steps and Layers for Accelerating Music Generationcited by 16
  19. 2024Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language Models via Instruction Tuningcited by 15
  20. 2024PDMX: A Large-Scale Public Domain MusicXML Dataset for Symbolic Music Processingcited by 15
  21. 2025Benchmarking Music Generation Models and Metrics via Human Preference Studiescited by 15
  22. 2024COCOLA: Coherence-Oriented Contrastive Learning of Musical Audio Representationscited by 14
  23. 2024VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videoscited by 14
  24. 2024MusicFlow: Cascaded Flow Matching for Text Guided Music Generationcited by 14
  25. 2025Fast Text-to-Audio Generation with Adversarial Post-Trainingcited by 14
  26. 2024High Fidelity Text-Guided Music Editing via Single-Stage Flow Matchingcited by 13
  27. 2024LoVA: Long-form Video-to-Audio Generationcited by 13
  28. 2024Editing Music with Melody and Text: Using ControlNet for Diffusion Transformercited by 13
  29. 2025Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generationcited by 13
  30. 2025Zero-Shot Audio-Visual Editing via Cross-Modal Delta Denoisingcited by 13
  31. 2025Audio Super-Resolution with Latent Bridge Modelscited by 13
  32. 2024SongCreator: Lyrics-based Universal Song Generationcited by 12
  33. 2025RayFlow: Instance-Aware Diffusion Acceleration via Adaptive Flow Trajectoriescited by 12
  34. 2024SpotDiffusion: A Fast Approach For Seamless Panorama Generation Over Timecited by 11
  35. 2024SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformercited by 11
  36. 2024ImmerseDiffusion: A Generative Spatial Audio Latent Diffusion Modelcited by 11
  37. 2024Improving Musical Accompaniment Co-creation via Diffusion Transformerscited by 11
  38. 2025Guiding Audio Editing with Audio Language Modelcited by 11

50 most cited of 178 All 178 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023ArchiSound: Audio Generation with Diffusion
  2. 2024Bass Accompaniment Generation via Latent Diffusion
  3. 2026Pitch-class Steering for Diffusion-based Music Generation via Latent-space Probes
  4. 2023Mousai: Text-to-Music Generation with Long-Context Latent Diffusion
  5. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementation
  6. 2026SwiftAudio: Data-Efficient Caption-Only Distillation for One-Step Text-to-Audio Diffusion-based Generation
  7. 2023EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis
  8. 2024AudioLCM: Text-to-Audio Generation with Latent Consistency Models