# Nonlinear Sufficient Dimension Reduction with a Stochastic Neural Network

arXiv: https://arxiv.org/abs/2210.04349 · HTML: https://mlsanity.com/paper/2210.04349/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 58 references that are in the map.
- First posted on arXiv 9 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis](https://mlsanity.com/paper/1702.03849/) (2017)
- [Adding Gradient Noise Improves Learning for Very Deep Networks](https://mlsanity.com/paper/1511.06807/) (2015)
- [An Improved Analysis of Training Over-parameterized Deep Neural Networks](https://mlsanity.com/paper/1906.04688/) (2019)
- [The loss surface of deep and wide neural networks](https://mlsanity.com/paper/1704.08045/) (2017)
- [Consistency and fluctuations for stochastic gradient Langevin dynamics](https://mlsanity.com/paper/1409.0578/) (2014)
- [Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization](https://mlsanity.com/paper/1710.05179/) (2017)
- [Stochastic gradient Markov chain Monte Carlo](https://mlsanity.com/paper/1907.06986/) (2019)
- [Noisy Activation Functions](https://mlsanity.com/paper/1603.00391/) (2016)
- [Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration](https://mlsanity.com/paper/1809.04618/) (2018)
- [Consistent Sparse Deep Learning: Theory and Computation](https://mlsanity.com/paper/2102.13229/) (2021)
- [A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions](https://mlsanity.com/paper/2010.09800/) (2020)
- [Adversarial Noise Layer: Regularize Neural Network By Adding Noise](https://mlsanity.com/paper/1805.08000/) (2018)
- [An Adaptive Empirical Bayesian Method for Sparse Deep Learning](https://mlsanity.com/paper/1910.10791/) (2019)
- [A Kernel-Expanded Stochastic Neural Network](https://mlsanity.com/paper/2201.05319/) (2022)
- [Sparse Sliced Inverse Regression Via Lasso](https://mlsanity.com/paper/1611.06655/) (2016)
- [Conditional Variance Estimator for Sufficient Dimension Reduction](https://mlsanity.com/paper/2102.08782/) (2021)
- [Deep Variational Sufficient Dimensionality Reduction](https://mlsanity.com/paper/1812.07641/) (2018)

## Built on by (most cited first, 10 of 10)

- [Exploring Loss Landscapes through the Lens of Spin Glass Theory](https://mlsanity.com/paper/2407.20724/) (2024)
- [Belted and Ensembled Neural Network for Linear and Nonlinear Sufficient Dimension Reduction](https://mlsanity.com/paper/2412.08961/) (2024)
- [Causal-StoNet: Causal Inference for High-Dimensional Complex Data](https://mlsanity.com/paper/2403.18994/) (2024)
- [Uncertainty Quantification for Physics-Informed Neural Networks with Extended Fiducial Inference](https://mlsanity.com/paper/2505.19136/) (2025)
- [Golden Ratio-Based Sufficient Dimension Reduction](https://mlsanity.com/paper/2410.19300/) (2024)
- [Magnitude Pruning of Large Pretrained Transformer Models with a Mixture Gaussian Prior](https://mlsanity.com/paper/2411.00969/) (2024)
- [Extended Fiducial Inference for Individual Treatment Effects via Deep Neural Networks](https://mlsanity.com/paper/2505.01995/) (2025)
- [Uncertainty Quantification for Large-Scale Deep Networks via Post-StoNet Modeling](https://mlsanity.com/paper/2508.01217/) (2025)
- [FlowSDR: Sufficient Dimension Reduction via Conditional Normalizing Flows](https://mlsanity.com/paper/2606.01346/) (2026)
- [Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions](https://mlsanity.com/paper/2606.23477/) (2026)

## Abstract (arXiv)

Sufficient dimension reduction is a powerful tool to extract core information hidden in the high-dimensional data and has potentially many important applications in machine learning tasks. However, the existing nonlinear sufficient dimension reduction methods often lack the scalability necessary for dealing with large-scale data. We propose a new type of stochastic neural network under a rigorous probabilistic framework and show that it can be used for sufficient dimension reduction for large-scale data. The proposed stochastic neural network is trained using an adaptive stochastic gradient Markov chain Monte Carlo algorithm, whose convergence is rigorously studied in the paper as well. Through extensive experiments on real-world classification and regression problems, we show that the proposed method compares favorably with the existing state-of-the-art sufficient dimension reduction methods and is computationally more efficient for large-scale data.