# DDSP-SFX: Acoustically-guided sound effects generation with differentiable digital signal processing

arXiv: https://arxiv.org/abs/2309.08060 · HTML: https://mlsanity.com/paper/2309.08060/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2023.

## Built on by (most cited first, 5 of 5)

- [Sines, Transient, Noise Neural Modeling of Piano Notes](https://mlsanity.com/paper/2409.06513/) (2024)
- [When Humans Growl and Birds Speak: High-Fidelity Voice Conversion from Human to Animal and Designed Sounds](https://mlsanity.com/paper/2505.24336/) (2025)
- [MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixing](https://mlsanity.com/paper/2507.06329/) (2025)
- [TimberAgent: Gram-Guided Retrieval for Executable Music Effect Control](https://mlsanity.com/paper/2603.09332/) (2026)
- [A Production-Oriented Framework for Evaluation of SFX Generation](https://mlsanity.com/paper/2607.09973/) (2026)

## Abstract (arXiv)

Controlling the variations of sound effects using neural audio synthesis models has been a difficult task. Differentiable digital signal processing (DDSP) provides a lightweight solution that achieves high-quality sound synthesis while enabling deterministic acoustic attribute control by incorporating pre-processed audio features and digital synthesizers. In this research, we introduce DDSP-SFX, a model based on the DDSP architecture capable of synthesizing high-quality sound effects while enabling users to control the timbre variations easily. We propose a transient modelling technique with higher objective evaluation scores and subjective ratings over impulsive signals (footsteps, gunshots). We propose a simple method that achieves timbre variation control while also allowing deterministic attribute control. We further qualitatively show the timbre transfer performance using voice as the guiding sound.