mlsanityData to 7 Oct 2026

Efficient Learning of Generative Models via Finite-Difference Score Matching

arXiv 2007.03317 ↗First posted 7 Jul 2020cs.LG stat.MLcited by 49

Not on the map. Why?

Cited by 23 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Several machine learning applications involve the optimization of higher-order derivatives (e.g., gradients of gradients) during training, which can be expensive in respect to memory and computation even with automatic differentiation. As a typical example in generative modeling, score matching (SM) involves the optimization of the trace of a Hessian. To improve computing efficiency, we rewrite the SM objective and its variants in terms of directional derivatives, and present a generic strategy to efficiently approximate any-order directional derivative with finite difference (FD). Our approximation only involves function evaluations, which can be executed in parallel, and no gradient computations. Thus, it reduces the total computational cost while also improving numerical stability. We provide two instantiations by reformulating variants of SM objectives into the FD forms. Empirically, we demonstrate that our methods produce results comparable to the gradient-based counterparts while being much more computationally efficient.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 3; 2022: 7; 2023: 13; 2024: 16; 2025: 7; 2026: 2.

49 papers.

Sort:

By area:

  1. 2021Score-based Generative Modeling in Latent Spacecited by 634
  2. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  3. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  4. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
  5. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  6. 2024Accelerating Convergence of Score-Based Diffusion Models, Provablycited by 74
  7. 2021Estimating High Order Gradients of the Data Distribution by Denoisingcited by 63
  8. 2023How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Controlcited by 50
  9. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  10. 2023A Comprehensive Survey on Knowledge Distillation of Diffusion Modelscited by 45
  11. 2024Attacks and Defenses for Generative Diffusion Models: A Comprehensive Surveycited by 32
  12. 2024Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equationscited by 28
37 more
  1. 2022Accelerated Training of Physics-Informed Neural Networks (PINNs) using Meshless Discretizationscited by 26
  2. 2020Diffusion models for Handwriting Generationcited by 25
  3. 2024The Rise of Diffusion Models in Time-Series Forecastingcited by 25
  4. 2022DiffMD: A Geometric Diffusion Model for Molecular Dynamics Simulationscited by 24
  5. 2022Learning Physics-Informed Neural Networks without Stacked Back-propagationcited by 23
  6. 2024Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equationscited by 22
  7. 2024Instance-dependent Convergence Theory for Diffusion Modelscited by 21
  8. 2024Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operatorscited by 20
  9. 2023Enhancing Adversarial Robustness via Score-Based Optimizationcited by 19
  10. 2025Optimal Convergence Analysis of DDPM for General Distributionscited by 16
  11. 2023Entropy-based Training Methods for Scalable Neural Implicit Samplercited by 13
  12. 2023Purify++: Improving Diffusion-Purification with Advanced Diffusion Models and Control of Randomnesscited by 13
  13. 2023BAFFLE: A Baseline of Backpropagation-Free Federated Learningcited by 12
  14. 2024Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Surveycited by 9
  15. 2025Provable Efficiency of Guidance in Diffusion Models for General Data Distributioncited by 8
  16. 2024Identifiability Guarantees for Causal Disentanglement from Purely Observational Datacited by 7
  17. 2023Training Energy-Based Normalizing Flow with Score-Matching Objectivescited by 6
  18. 2025Diffusion Generative Modeling on Lie Group Representationscited by 6
  19. 2022MARS: Meta-Learning as Score Matching in the Function Spacecited by 5
  20. 2023Learning Unnormalized Statistical Models via Compositional Optimizationcited by 5
  21. 2024DOF: Accelerating High-order Differential Operators with Forward Propagationcited by 5
  22. 2024Moderating the Generalization of Score-based Generative Modelcited by 4
  23. 2023On Calibrating Diffusion Probabilistic Modelscited by 3
  24. 2025Beyond Scores: Proximal Diffusion Modelscited by 3
  25. 2025Agentic Satellite-Augmented Low-Altitude Economy and Terrestrial Networks: A Survey on Generative Approachescited by 2
  26. 2022Self-Adapting Noise-Contrastive Estimation for Energy-Based Modelscited by 1
  27. 2023Approximate Stein Classes for Truncated Density Estimationcited by 1
  28. 2024Efficient Denoising using Score Embedding in Score-based Diffusion Modelscited by 1
  29. 2024Score-fPINN: Fractional Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck-Levy Equationscited by 1
  30. 2025Incorporating Inductive Biases to Energy-based Generative Modelscited by 1
  31. 2025Efficient Score Pre-computation for Diffusion Models via Cross-Matrix Krylov Projectioncited by 1
  32. 2023A Score-Based Model for Learning Neural Wavefunctionscited by 0
  33. 2024Combinatorial Complex Score-based Diffusion Modelling through Stochastic Differential Equationscited by 0
  34. 2024Denoising Fisher Training For Neural Implicit Samplerscited by 0
  35. 2024Local Curvature Smoothing with Stein's Identity for Efficient Score Matchingcited by 0
  36. 2026Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNscited by 0
  37. 2026Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyondcited by 0