mlsanityData to 7 Oct 2026

A Review of Differentiable Digital Signal Processing for Music & Speech Synthesis

arXiv 2308.15422 ↗First posted 29 Aug 2023cs.SD eess.AScited by 19

Not on the map. Why?

Abstract (arXiv)

The term "differentiable digital signal processing" describes a family of techniques in which loss function gradients are backpropagated through digital signal processors, facilitating their integration into neural networks. This article surveys the literature on differentiable audio signal processing, focusing on its use in music & speech synthesis. We catalogue applications to tasks including music performance rendering, sound matching, and voice transformation, discussing the motivations for and implications of the use of this methodology. This is accompanied by an overview of digital signal processing operations that have been implemented differentiably. Finally, we highlight open challenges, including optimisation pathologies, robustness to real-world conditions, and design trade-offs, and discuss directions for future research.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Aug 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 8; 2026: 6.

19 papers.

Sort:

By area:

  1. 2023Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transportcited by 8
  2. 2025Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matchingcited by 6
  3. 2025TokenSynth: A Token-based Neural Synthesizer for Instrument Cloning and Text-to-Instrumentcited by 5
  4. 2024Differentiable Modal Synthesis for Physical Modeling of Planar String Sound and Motion Simulationcited by 4
  5. 2024Real-time Timbre Remapping with Differentiable DSPcited by 3
  6. 2024Fast, High-Quality and Parameter-Efficient Articulatory Synthesis using Differentiable DSPcited by 3
  7. 2025The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesiscited by 3
  8. 2025PESTO: Real-Time Pitch Estimation with Self-supervised Transposition-equivariant Objectivecited by 3
  9. 2025DiTSinger: Scaling Singing Voice Synthesis with Diffusion Transformer and Implicit Alignmentcited by 3
  10. 2024Generating Sample-Based Musical Instruments Using Neural Audio Codec Language Modelscited by 1
  11. 2025Designing Neural Synthesizers for Low-Latency Interactioncited by 1
  12. 2025Modulation Discovery with Differentiable Digital Signal Processingcited by 1
7 more
  1. 2025Spectrotemporal Modulation: Efficient and Interpretable Feature Representation for Classifying Speech, Music, and Environmental Soundscited by 0
  2. 2026Gradient-based Optimisation of Modulation Effectscited by 0
  3. 2026Lightweight Self-Supervised Detection of Fundamental Frequency and Accurate Probability of Voicing in Monophonic Musiccited by 0
  4. 2026SCRAPL: Scattering Transform with Random Paths for Machine Learningcited by 0
  5. 2026Architecture and Affordances of PLAUD: Performative Latents and Unsupervised DDSPcited by 0
  6. 2026Differentiable Digital Signal Processing Mixture Model-Guided Diffusion for Synthesis Parameter Estimation from Harmonic Sound Mixturescited by 0
  7. 2026Off-manifold robustness in synthesizer inversion with joint distribution flow matchingcited by 0