SoundLoCD: An Efficient Conditional Discrete Contrastive Latent Diffusion Model for Text-to-Sound Generation
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
We present SoundLoCD, a novel text-to-sound generation framework, which incorporates a LoRA-based conditional discrete contrastive latent diffusion model. Unlike recent large-scale sound generation models, our model can be efficiently trained under limited computational resources. The integration of a contrastive learning strategy further enhances the connection between text conditions and the generated outputs, resulting in coherent and high-fidelity performance. Our experiments demonstrate that SoundLoCD outperforms the baseline with greatly reduced computational resources. A comprehensive ablation study further validates the contribution of each component within SoundLoCD. Demo page: urlhttps://XinleiNIU.github.io/demo-SoundLoCD/.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 21 of its 24 references that are in the map. First posted on arXiv 24 May 2024.
Builds on
21 of 24 references.
Closest ancestors · 7
Same direction, newest first
- 2023Text-Driven Foley Sound Generation With Latent Diffusion Model17 Jun
- 2023Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation29 May
- 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model24 Apr
- 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models30 Jan
- 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan
- 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generation20 Jul
- 2022Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation15 Jun
Other references · 14
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2017Neural Discrete Representation Learningcited by 4,712
- 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
2 more
- 2022AudioGen: Textually Guided Audio Generationcited by 295
- 2020Real-time Speech Frequency Bandwidth Extensioncited by 22
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 2.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2024LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation
- 2024PTQ4ADM: Post-Training Quantization for Efficient Text Conditional Audio Diffusion Models
- 2026UAT: Unified Audio-Text Diffusion for Audio Generation, Editing, and Captioning
- 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation
- 2026SwiftAudio: Data-Efficient Caption-Only Distillation for One-Step Text-to-Audio Diffusion-based Generation
- 2024AudioLCM: Text-to-Audio Generation with Latent Consistency Models
- 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementation
- 2025Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis