mlsanityData to 7 Oct 2026

Concrete Score Matching: Generalized Score Matching for Discrete Data

arXiv 2211.00802 ↗First posted 2 Nov 2022cs.LG cs.AIcited by 144

Not on the map. Why?

Cited by 108 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Representing probability distributions by the gradient of their density functions has proven effective in modeling a wide range of continuous data modalities. However, this representation is not applicable in discrete domains where the gradient is undefined. To this end, we propose an analogous score function called the "Concrete score", a generalization of the (Stein) score for discrete settings. Given a predefined neighborhood structure, the Concrete score of any input is defined by the rate of change of the probabilities with respect to local directional changes of the input. This formulation allows us to recover the (Stein) score in continuous domains when measuring such changes by the Euclidean distance, while using the Manhattan distance leads to our novel score function in discrete domains. Finally, we introduce a new framework to learn such scores from samples called Concrete Score Matching (CSM), and propose an efficient training objective to scale our approach to high dimensions. Empirically, we demonstrate the efficacy of CSM on density estimation tasks on a mixture of synthetic, tabular, and high-dimensional image datasets, and demonstrate that it performs favorably relative to existing baselines for modeling discrete data.

Key facts. Cited by 144 papers in the map (data through 7 Oct 2026). Builds on 48 of its 82 references that are in the map. First posted on arXiv 2 Nov 2022.

Builds on

48 of 82 references.

References · 48

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2015Variational Inference with Normalizing Flowscited by 2,161
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
36 more
  1. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  3. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  4. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  5. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  6. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  7. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  8. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  9. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  10. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  11. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  12. 2020Permutation Invariant Graph Generation via Score-Based Generative Modelingcited by 263
  13. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  14. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translationcited by 212
  15. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
  16. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  17. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  18. 2019Discrete Flows: Invertible Generative Models of Discrete Datacited by 68
  19. 2019Latent Normalizing Flows for Discrete Sequencescited by 66
  20. 2021Estimating High Order Gradients of the Data Distribution by Denoisingcited by 63
  21. 2019Block Neural Autoregressive Flowcited by 62
  22. 2021Density Ratio Estimation via Infinitesimal Classificationcited by 59
  23. 2020Accelerating Feedforward Computation via Parallel Nonlinear Equation Solvingcited by 51
  24. 2020Efficient Learning of Generative Models via Finite-Difference Score Matchingcited by 49
  25. 2019MintNet: Building Invertible Neural Networks with Masked Convolutionscited by 35
  26. 2019SPFlow: An Easy and Extensible Library for Deep Probabilistic Learning using Sum-Product Networkscited by 26
  27. 2020Gaussianization Flowscited by 20
  28. 2014Expectation-Maximization for Learning Determinantal Point Processescited by 19
  29. 2014Accurate and Conservative Estimates of MRF Log-likelihood using Reverse Annealingcited by 18
  30. 2021Improved Autoregressive Modeling with Distribution Smoothingcited by 18
  31. 2020Autoregressive Score Matchingcited by 15
  32. 2022ButterflyFlow: Building Invertible Layers with Butterfly Matricescited by 12
  33. 2020Bi-level Score Matching for Learning Energy-based Latent Variable Modelscited by 10
  34. 2020Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Modelscited by 8
  35. 2022Self-Similarity Priors: Neural Collages as Differentiable Fractal Representationscited by 6
  36. 2022Gradient-Guided Importance Sampling for Learning Binary Energy-Based Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 10; 2024: 30; 2025: 59; 2026: 44.

144 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 872
  2. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  3. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  4. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  5. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  6. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  7. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  8. 2022Continuous diffusion for categorical datacited by 202
  9. 2024Scaling up Masked Diffusion Models on Textcited by 195
  10. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  11. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  12. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
38 more
  1. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  2. 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
  3. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  4. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  5. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Modelscited by 67
  6. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  7. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  8. 2024Discrete Copula Diffusioncited by 53
  9. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  10. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  11. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  12. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 46
  13. 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformizationcited by 44
  14. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  15. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  16. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  17. 2023Sampling, Diffusions, and Stochastic Localizationcited by 38
  18. 2023Protein Discovery with Discrete Walk-Jump Samplingcited by 38
  19. 2023MissDiff: Training Diffusion Models on Tabular Data with Missing Valuescited by 38
  20. 2026Categorical Flow Mapscited by 37
  21. 2025Fine-Tuning Discrete Diffusion Models with Policy Gradient Methodscited by 36
  22. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  23. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 28
  24. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  25. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysiscited by 24
  26. 2025Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Drivingcited by 24
  27. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  28. 2025Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Modelscited by 20
  29. 2025DLM-One: Diffusion Language Models for One-Step Sequence Generationcited by 19
  30. 2023Binary Latent Diffusioncited by 18
  31. 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approachcited by 18
  32. 2023Continuous Diffusion for Mixed-Type Tabular Datacited by 16
  33. 2025Constrained Discrete Diffusioncited by 16
  34. 2026Continuous Diffusion Scales Competitively with Discrete Diffusion for Languagecited by 13
  35. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 12
  36. 2026dLLM: Simple Diffusion Language Modelingcited by 12
  37. 2024Glauber Generative Model: Discrete Diffusion Models via Binary Classificationcited by 11
  38. 2024Promises, Outlooks and Challenges of Diffusion Language Modelingcited by 11

50 most cited of 144 All 144 (JSON)