mlsanityData to 7 Oct 2026

Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion

arXiv 2402.14285 ↗First posted 22 Feb 2024cs.SD cs.LG eess.AScited by 52

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We study the problem of symbolic music generation (e.g., generating piano rolls), with a technical focus on non-differentiable rule guidance. Musical rules are often expressed in symbolic form on note characteristics, such as note density or chord progression, many of which are non-differentiable which pose a challenge when using them for guided diffusion. We propose Stochastic Control Guidance (SCG), a novel guidance method that only requires forward evaluation of rule functions that can work with pre-trained diffusion models in a plug-and-play way, thus achieving training-free guidance for non-differentiable rules for the first time. Additionally, we introduce a latent diffusion architecture for symbolic music generation with high time resolution, which can be composed with SCG in a plug-and-play fashion. Compared to standard strong baselines in symbolic music generation, this framework demonstrates marked advancements in music quality and rule-based controllability, outperforming current state-of-the-art generators in a variety of settings. For detailed demonstrations, code and model checkpoints, please visit our project website: https://scg-rule-guided-music.github.io/.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 32 of its 52 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

32 of 52 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controls19 Jul
  2. 2023Generating symbolic music using diffusion models15 Mar
  3. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Models8 Feb

Other references · 29

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  11. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  12. 2022Denoising Diffusion Restoration Modelscited by 951
17 more
  1. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  2. 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
  3. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  4. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
  5. 2022An optimal control perspective on diffusion-based generative modelingcited by 145
  6. 2023Denoising Diffusion Samplerscited by 144
  7. 2018A Hierarchical Latent Vector Model for Learning Long-Term Structure in Musiccited by 131
  8. 2021Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphscited by 109
  9. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 100
  10. 2020POP909: A Pop-song Dataset for Music Arrangement Generationcited by 83
  11. 2017MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generationcited by 72
  12. 2020PopMAG: Pop Music Accompaniment Generationcited by 68
  13. 2021MuseMorphose: Full-Song and Fine-Grained Piano Music Style Transfer with One Transformer VAEcited by 39
  14. 2019Encoding Musical Style with Transformer Autoencoderscited by 35
  15. 2018MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfercited by 33
  16. 2022SDMuse: Stochastic Differential Music Editing and Generation via Hybrid Representationcited by 11
  17. 2019Exploring Conditioning for Generative Music Systems with Human-Interpretable Controlscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 22; 2026: 17.

52 papers.

Sort:

By area:

  1. 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Controlcited by 222
  2. 2025InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciencescited by 67
  3. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  4. 2025Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Searchcited by 40
  5. 2025Reangle-A-Video: 4D Video Generation as Video-to-Video Translationcited by 35
  6. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  7. 2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 23
  8. 2025Variational Control for Guidance in Diffusion Modelscited by 22
  9. 2024Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problemscited by 18
  10. 2025Split Gibbs Discrete Diffusion Posterior Samplingcited by 18
  11. 2025Neurosymbolic Diffusion Modelscited by 12
  12. 2025Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generationcited by 11
38 more
  1. 2025HardFlow: Hard-Constrained Sampling for Flow-Matching Models via Trajectory Optimizationcited by 10
  2. 2024Plug-and-Play Controllable Generation for Discrete Masked Modelscited by 9
  3. 2024Free2Guide: Training-Free Text-to-Video Alignment using Image LVLMcited by 9
  4. 2024Neural Approximate Mirror Maps for Constrained Diffusion Modelscited by 7
  5. 2024Chemistry-Inspired Diffusion with Non-Differentiable Guidancecited by 6
  6. 2025Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpeningcited by 6
  7. 2025Graph Guided Diffusion: Unified Guidance for Conditional Graph Generationcited by 5
  8. 2025Tree-Guided Diffusion Plannercited by 4
  9. 2026Calibrated Test-Time Guidance for Bayesian Inferencecited by 4
  10. 2025Mamba-Diffusion Model with Learnable Wavelet for Controllable Symbolic Music Generationcited by 3
  11. 2025Blade: A Derivative-free Bayesian Inversion Method using Diffusion Priorscited by 3
  12. 2024SYMPLEX: Controllable Symbolic Music Generation using Simplex Diffusion with Vocabulary Priorscited by 2
  13. 2024BandCondiNet: Parallel Transformers-based Conditional Popular Music Generation with Multi-View Featurescited by 2
  14. 2025System 0/1/2/3: Quad-process theory for multi-timescale embodied collective cognitive systemscited by 2
  15. 2025Diffusion-based Symbolic Music Generation with Structured State Space Modelscited by 2
  16. 2024Prevailing Research Areas for Music AI in the Era of Foundation Modelscited by 1
  17. 2024Efficient Fine-Grained Guidance for Diffusion Model Based Symbolic Music Generationcited by 1
  18. 2025Not that Groove: Zero-Shot Symbolic Music Editingcited by 1
  19. 2025ProGress: Structured Music Generation via Graph Diffusion and Hierarchical Music Analysiscited by 1
  20. 2026Constrained Particle Seeking: Solving Diffusion Inverse Problems with Just Forward Passescited by 1
  21. 2026Gradient-Free Noise Optimization for Reward Alignment in Generative Modelscited by 1
  22. 2026BeatEdit: Symbolic Music Generation as Explicit Editingcited by 1
  23. 2024Reinforcement Learning Jazz Improvisation: When Music Meets Game Theorycited by 0
  24. 2024PerceiverS: A Multi-Scale Perceiver with Effective Segmentation for Long-Term Expressive Symbolic Music Generationcited by 0
  25. 2025TOMI: Transforming and Organizing Music Ideas for Multi-Track Compositions with Full-Song Structurecited by 0
  26. 2025Conditional Diffusion as Latent Constraints for Controllable Symbolic Music Generationcited by 0
  27. 2025DPAC: Distribution-Preserving Adversarial Control for Diffusion Samplingcited by 0
  28. 2026Pay (Cross) Attention to the Melody: Curriculum Masking for Single-Encoder Melodic Harmonizationcited by 0
  29. 2026Efficient Long-Sequence Diffusion Modeling for Symbolic Music Generationcited by 0
  30. 2026AMGenC: Generating Charge Balanced Amorphous Materialscited by 0
  31. 2026Hierarchical Variational Policies for Reward-Guided Diffusioncited by 0
  32. 2026Zeroth-Order Non-Log-Concave Sampling with Variance Reduction and Applications to Inverse Problemscited by 0
  33. 2026NDPP-Grasp: Non-Differentiable Physical Plausibility Constraint-Guided Task-Oriented Dexterous Grasp Generationcited by 0
  34. 2026Variational Test-time Optimization for Diffusion Synchronizationcited by 0
  35. 2026Histogram-constrained Image Generationcited by 0
  36. 2026Training-free Controllable Human Motion Generation under Heterogeneous Constraintscited by 0
  37. 2026Parallel Noising in Neural Markov Logic Networkscited by 0
  38. 2026Which Constraints Are Missing? Ask the Verifier: Graded Rewards for Constraint-Following Music Generationcited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Model
  2. 2025NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms
  3. 2024Why Perturbing Symbolic Music is Necessary: Fitting the Distribution of Never-used Notes through a Joint Probabilistic Diffusion Model
  4. 2025PhraseVAE and PhraseLDM: Latent Diffusion for Full-Song Multitrack Symbolic Music Generation
  5. 2023Composer Style-specific Symbolic Music Generation Using Vector Quantized Discrete Diffusion Models
  6. 2021Symbolic Music Generation with Diffusion Models
  7. 2023GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework
  8. 2024Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model