mlsanityData to 14 Oct 2026

A Spectral Approach to Gradient Estimation for Implicit Distributions

arXiv 1806.02925 ↗First posted 7 Jun 2018stat.ML cs.LG cs.NEcited by 45

Not on the map. Why?

Abstract (arXiv)

Recently there have been increasing interests in learning and inference with implicit distributions (i.e., distributions without tractable densities). To this end, we develop a gradient estimator for implicit distributions based on Stein's identity and a spectral decomposition of kernel operators, where the eigenfunctions are approximated by the Nystr"om method. Unlike the previous works that only provide estimates at the sample points, our approach directly estimates the gradient function, thus allows for a simple and principled out-of-sample extension. We provide theoretical results on the error bound of the estimator and discuss the bias-variance tradeoff in practice. The effectiveness of our method is demonstrated by applications to gradient-free Hamiltonian Monte Carlo and variational inference with implicit distributions. Finally, we discuss the intuition behind the estimator by drawing connections between the Nystr"om method and kernel PCA, which indicates that the estimator can automatically adapt to the geometry of the underlying distribution.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 7; 2023: 8; 2024: 9; 2025: 5; 2026: 5.

45 papers.

Sort:

By area:

  1. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  2. 2021Repulsive Deep Ensembles are Bayesiancited by 104
  3. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  4. 2022Diffusion Models for Causal Discovery via Topological Orderingcited by 66
  5. 2021Posterior Meta-Replay for Continual Learningcited by 45
  6. 2021LiBRe: A Practical Bayesian Approach to Adversarial Detectioncited by 40
  7. 2023A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methodscited by 32
  8. 2021Meta-Learning Reliable Priors in the Function Spacecited by 23
  9. 2021On Stein Variational Neural Network Ensemblescited by 21
  10. 2022Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learningcited by 21
  11. 2023Semi-Implicit Variational Inference via Score Matchingcited by 19
  12. 2021Sampling with Mirrored Stein Operatorscited by 17
33 more
  1. 2023Nonparametric Generative Modeling with Conditional Sliced-Wasserstein Flowscited by 15
  2. 2023Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Accelerationcited by 14
  3. 2021Bounds all around: training energy-based models with bidirectional boundscited by 10
  4. 2022Feature Space Particle Inference for Neural Network Ensemblescited by 9
  5. 2023Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regressioncited by 9
  6. 2022Deep Variational Implicit Processescited by 8
  7. 2024Bridging the Sim-to-Real Gap with Bayesian Inferencecited by 8
  8. 2024Kernel Semi-Implicit Variational Inferencecited by 8
  9. 2024A Skewness-Based Criterion for Addressing Heteroscedastic Noise in Causal Discoverycited by 7
  10. 2022MARS: Meta-Learning as Score Matching in the Function Spacecited by 5
  11. 2023On Estimating the Gradient of the Expected Information Gain in Bayesian Experimental Designcited by 5
  12. 2024Low-Budget Simulation-Based Inference with Bayesian Neural Networkscited by 5
  13. 2021Active Slices for Sliced Stein Discrepancycited by 3
  14. 2021AutoTransfer: Subject Transfer Learning with Censored Representations on Biosignals Datacited by 3
  15. 2024Nonparametric Automatic Differentiation Variational Inference with Spline Approximationcited by 3
  16. 2024Functional Stochastic Gradient MCMC for Bayesian Neural Networkscited by 2
  17. 2024Revisiting the Equivalence of Bayesian Neural Networks and Gaussian Processes: On the Importance of Learning Activationscited by 2
  18. 2025Causal Ordering for Structure Learning from Time Seriescited by 2
  19. 2021Accurate and Reliable Forecasting using Stochastic Differential Equationscited by 1
  20. 2022Augmenting Neural Networks with Priors on Function Valuescited by 1
  21. 2022Deep Ensemble as a Gaussian Process Approximate Posteriorcited by 1
  22. 2025Nonlinear Multiple Response Regression and Learning of Latent Spacescited by 1
  23. 2026A Kernel Approach for Semi-implicit Variational Inferencecited by 1
  24. 2026Multi-Source Transfer Learning of Sparse Single-Index Modelscited by 1
  25. 2026Bayesian Experimental Design via Score Matchingcited by 1
  26. 2023Function-Space Regularization for Deep Bayesian Classificationcited by 0
  27. 2024Local Curvature Smoothing with Stein's Identity for Efficient Score Matchingcited by 0
  28. 2024Score Change of Variablescited by 0
  29. 2025Exploring bidirectional bounds for minimax-training of Energy-based modelscited by 0
  30. 2025Simulation Priors for Data-Efficient Deep Learningcited by 0
  31. 2025Latent Uncertainty Representations for Video-based Driver Action and Intention Recognitioncited by 0
  32. 2026Function-Space Empirical Bayes Regularisation with Large Vision-Language Model Priorscited by 0
  33. 2026Function-Space Empirical Bayes Regularisation with Student's t Priorscited by 0