mlsanityData to 7 Oct 2026

Seconds-Aligned PCA-DAC Latent Diffusion for Symbolic-to-Audio Drum Rendering

arXiv 2605.13404 ↗First posted 13 May 2026cs.SDcited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Symbolic-control drum generation requires preserving explicit event timing and dynamics while synthesizing acoustically plausible waveforms. We present Sec2Drum-DAC, a conditional latent-diffusion model for symbolic-to-audio drum rendering. The model conditions on event features sampled in physical time at codec-frame locations and predicts standardized principal-component coordinates of frozen DAC summed-codebook embeddings rather than waveform samples. In the evaluated DAC configuration, 72 principal components capture the observed training-frame summed-latent subspace under the stated SVD threshold, yielding a compact continuous denoising target with a deterministic reconstruction path to the 1024-dimensional DAC latent space before waveform decoding. Across 1,733 held-out four-beat windows, PCA diffusion improves paired spectral and transient metrics over deterministic PCA regression and a symbolic rendering baseline, while direct regression remains stronger on phase-sensitive waveform L1. Auxiliary RVQ cross-entropy improves short-step diffusion on mel error, onset-flux cosine, and waveform L1, with the most favorable trade-offs occurring at 6-25 denoising steps depending on the metric.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 29 of its 36 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

29 of 36 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Text Conditioned Symbolic Drumbeat Generation using Latent Diffusion Models5 Aug
  2. 2024Long-form music generation with latent diffusion16 Apr
  3. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
  4. 2022Multi-instrument Music Synthesis with Spectrogram Diffusion11 Jun
  5. 2021CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis14 Jun

Other references · 24

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  6. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  7. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  8. 2022High Fidelity Neural Audio Compressioncited by 673
  9. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  10. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  11. 2023MusicLM: Generating Music From Textcited by 442
  12. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
12 more
  1. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  2. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  3. 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 176
  4. 2023Adapting Frechet Audio Distance for Generative Music Evaluationcited by 92
  5. 2016madmom: a new Python Audio and Music Signal Processing Librarycited by 63
  6. 2020DrumGAN: Synthesis of Drum Sounds With Timbral Feature Conditioning Using Generative Adversarial Networkscited by 34
  7. 2019Learning to Groove with Inverse Sequence Transformationscited by 33
  8. 2025Benchmarking Music Generation Models and Metrics via Human Preference Studiescited by 15
  9. 2025MIDI-VALLE: Improving Expressive Piano Performance Synthesis Through Neural Codec Language Modellingcited by 8
  10. 2025The Rhythm In Anything: Audio-Prompted Drums Generation with Masked Language Modelingcited by 3
  11. 2026DARC: Drum accompaniment generation with fine-grained rhythm controlcited by 2
  12. 2026Drum Synthesis from Expressive Drum Grids via Neural Audio Codecscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Noise-to-Notes: Diffusion-based Generation and Refinement for Automatic Drum Transcription
  2. 2026Separate-and-Detect: Unified Drum Transcription and Stem Generation via Latent Diffusion
  3. 2026A Cold Diffusion Approach for Percussive Dereverberation
  4. 2026TempoSyncDiff: Distilled Temporally-Consistent Diffusion for Low-Latency Audio-Driven Talking Head Generation
  5. 2026Low-Resource Guidance for Controllable Latent Audio Diffusion
  6. 2023EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis
  7. 2025D3RM: A Discrete Denoising Diffusion Refinement Model for Piano Transcription
  8. 2026DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning