# Active Slices for Sliced Stein Discrepancy

arXiv: https://arxiv.org/abs/2102.03159 · HTML: https://mlsanity.com/paper/2102.03159/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 5 Feb 2021.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [Generalized Sliced Wasserstein Distances](https://mlsanity.com/paper/1902.00434/) (2019)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [Max-Sliced Wasserstein Distance and its use for GANs](https://mlsanity.com/paper/1904.05877/) (2019)
- [Measuring Sample Quality with Stein's Method](https://mlsanity.com/paper/1506.03039/) (2015)
- [Statistical and Topological Properties of Sliced Probability Divergences](https://mlsanity.com/paper/2003.05783/) (2020)
- [Gradient Estimators for Implicit Models](https://mlsanity.com/paper/1705.07107/) (2017)
- [A Spectral Approach to Gradient Estimation for Implicit Distributions](https://mlsanity.com/paper/1806.02925/) (2018)
- [Density Estimation in Infinite Dimensional Exponential Families](https://mlsanity.com/paper/1312.3516/) (2013)
- [A Linear-Time Kernel Goodness-of-Fit Test](https://mlsanity.com/paper/1705.07673/) (2017)
- [Learning deep kernels for exponential family densities](https://mlsanity.com/paper/1811.08357/) (2018)
- [Certified dimension reduction in nonlinear Bayesian inverse problems](https://mlsanity.com/paper/1807.03712/) (2018)
- [Conditional Noise-Contrastive Estimation of Unnormalised Models](https://mlsanity.com/paper/1806.03664/) (2018)
- [Random Feature Stein Discrepancies](https://mlsanity.com/paper/1806.07788/) (2018)
- [Nonparametric Score Estimators](https://mlsanity.com/paper/2005.10099/) (2020)
- [Sliced Kernelized Stein Discrepancy](https://mlsanity.com/paper/2006.16531/) (2020)
- [Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions](https://mlsanity.com/paper/1901.08659/) (2019)
- [Efficient and principled score estimation with Nystr"om kernel exponential families](https://mlsanity.com/paper/1705.08360/) (2017)
- [Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling](https://mlsanity.com/paper/2002.05616/) (2020)
- [Kernelized Complete Conditional Stein Discrepancy](https://mlsanity.com/paper/1904.04478/) (2019)

## Built on by (most cited first, 3 of 3)

- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [BayesDAG: Gradient-Based Posterior Inference for Causal Discovery](https://mlsanity.com/paper/2307.13917/) (2023)
- [Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancy](https://mlsanity.com/paper/2304.14762/) (2023)

## Abstract (arXiv)

Sliced Stein discrepancy (SSD) and its kernelized variants have demonstrated promising successes in goodness-of-fit tests and model learning in high dimensions. Despite their theoretical elegance, their empirical performance depends crucially on the search of optimal slicing directions to discriminate between two distributions. Unfortunately, previous gradient-based optimisation approaches for this task return sub-optimal results: they are computationally expensive, sensitive to initialization, and they lack theoretical guarantees for convergence. We address these issues in two steps. First, we provide theoretical results stating that the requirement of using optimal slicing directions in the kernelized version of SSD can be relaxed, validating the resulting discrepancy with finite random slicing directions. Second, given that good slicing directions are crucial for practical performance, we propose a fast algorithm for finding such slicing directions based on ideas of active sub-space construction and spectral decomposition. Experiments on goodness-of-fit tests and model learning show that our approach achieves both improved performance and faster convergence. Especially, we demonstrate a 14-80x speed-up in goodness-of-fit tests when comparing with gradient-based alternatives.