mlsanityData to 7 Oct 2026

Differentiable Modal Synthesis for Physical Modeling of Planar String Sound and Motion Simulation

arXiv 2407.05516 ↗First posted 7 Jul 2024eess.AS cs.AI cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

While significant advancements have been made in music generation and differentiable sound synthesis within machine learning and computer audition, the simulation of instrument vibration guided by physical laws has been underexplored. To address this gap, we introduce a novel model for simulating the spatio-temporal motion of nonlinear strings, integrating modal synthesis and spectral modeling within a neural network framework. Our model leverages physical properties and fundamental frequencies as inputs, outputting string states across time and space that solve the partial differential equation characterizing the nonlinear string. Empirical evaluations demonstrate that the proposed architecture achieves superior accuracy in string motion simulation compared to existing baseline architectures. The code and demo are available online.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 23 of its 66 references that are in the map. First posted on arXiv 7 Jul 2024.

Builds on

23 of 66 references.

References · 23

  1. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  2. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  3. 2018SDR - half-baked or well done?cited by 420
  4. 2018Adversarial Audio Synthesiscited by 222
  5. 2020DDSP: Differentiable Digital Signal Processingcited by 177
  6. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  7. 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 109
  8. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
  9. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  10. 2019Neural source-filter waveform models for statistical parametric speech synthesiscited by 46
  11. 2022NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesiscited by 38
  12. 2021MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modelingcited by 31
11 more
  1. 2023A Review of Differentiable Digital Signal Processing for Music & Speech Synthesiscited by 19
  2. 2020I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitchcited by 17
  3. 2022Sinusoidal Frequency Estimation by Gradient Descentcited by 15
  4. 2022DDX7: Differentiable FM Synthesis of Musical Instrument Soundscited by 13
  5. 2021Neural Waveshaping Synthesiscited by 12
  6. 2021Differentiable Artificial Reverberationcited by 11
  7. 2022Physical Modeling using Recurrent Neural Networks with Fast Convolutional Layerscited by 9
  8. 2023Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transportcited by 8
  9. 2022Rigid-Body Sound Synthesis with Differentiable Modal Resonatorscited by 5
  10. 2023NoiseBandNet: Controllable Time-Varying Neural Synthesis of Sound Effects Using Filterbankscited by 4
  11. 2023String Sound Synthesizer on GPU-accelerated Finite Difference Schemecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Fast Differentiable Modal Simulation of Non-linear Strings, Membranes, and Platescited by 3
  2. 2025Learning Nonlinear Dynamics in Physical Modelling Synthesis using Neural Ordinary Differential Equationscited by 2
  3. 2026Stable Differentiable Modal Synthesis for Learning Nonlinear Dynamicscited by 0
  4. 2026Identifying the nonlinear string dynamics with port-Hamiltonian neural networkscited by 0