mlsanityData to 14 Oct 2026

Nonlinear Sufficient Dimension Reduction with a Stochastic Neural Network

arXiv 2210.04349 ↗First posted 9 Oct 2022cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Sufficient dimension reduction is a powerful tool to extract core information hidden in the high-dimensional data and has potentially many important applications in machine learning tasks. However, the existing nonlinear sufficient dimension reduction methods often lack the scalability necessary for dealing with large-scale data. We propose a new type of stochastic neural network under a rigorous probabilistic framework and show that it can be used for sufficient dimension reduction for large-scale data. The proposed stochastic neural network is trained using an adaptive stochastic gradient Markov chain Monte Carlo algorithm, whose convergence is rigorously studied in the paper as well. Through extensive experiments on real-world classification and regression problems, we show that the proposed method compares favorably with the existing state-of-the-art sufficient dimension reduction methods and is computationally more efficient for large-scale data.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 21 of its 58 references that are in the map. First posted on arXiv 9 Oct 2022.

Builds on

21 of 58 references.

References · 21

  1. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  3. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  4. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
  5. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
  6. 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 145
  7. 2019An Improved Analysis of Training Over-parameterized Deep Neural Networkscited by 121
  8. 2017The loss surface of deep and wide neural networkscited by 74
  9. 2014Consistency and fluctuations for stochastic gradient Langevin dynamicscited by 65
  10. 2017Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimizationcited by 56
  11. 2019Stochastic gradient Markov chain Monte Carlocited by 56
  12. 2016Noisy Activation Functionscited by 53
9 more
  1. 2018Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Accelerationcited by 31
  2. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
  3. 2020A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributionscited by 26
  4. 2018Adversarial Noise Layer: Regularize Neural Network By Adding Noisecited by 25
  5. 2019An Adaptive Empirical Bayesian Method for Sparse Deep Learningcited by 20
  6. 2022A Kernel-Expanded Stochastic Neural Networkcited by 9
  7. 2016Sparse Sliced Inverse Regression Via Lassocited by 8
  8. 2021Conditional Variance Estimator for Sufficient Dimension Reductioncited by 4
  9. 2018Deep Variational Sufficient Dimensionality Reductioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 2.

10 papers.

  1. 2024Exploring Loss Landscapes through the Lens of Spin Glass Theorycited by 4
  2. 2024Belted and Ensembled Neural Network for Linear and Nonlinear Sufficient Dimension Reductioncited by 4
  3. 2024Causal-StoNet: Causal Inference for High-Dimensional Complex Datacited by 3
  4. 2025Uncertainty Quantification for Physics-Informed Neural Networks with Extended Fiducial Inferencecited by 2
  5. 2024Golden Ratio-Based Sufficient Dimension Reductioncited by 1
  6. 2024Magnitude Pruning of Large Pretrained Transformer Models with a Mixture Gaussian Priorcited by 1
  7. 2025Extended Fiducial Inference for Individual Treatment Effects via Deep Neural Networkscited by 1
  8. 2025Uncertainty Quantification for Large-Scale Deep Networks via Post-StoNet Modelingcited by 0
  9. 2026FlowSDR: Sufficient Dimension Reduction via Conditional Normalizing Flowscited by 0
  10. 2026Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functionscited by 0