Statistical Inference on Gradient Flows
Not on the map. Why?
Abstract (arXiv)
Gradient-based algorithms are central to modern statistical estimation, yet their statistical analysis is often restricted to fixed-time behavior, such as convergence to a population target or fluctuations at a prescribed iteration. In many applications, however, uncertainty quantification is needed along the entire optimization path, especially when the stopping time is data-dependent or divergent. In this paper, we develop a theory for time-uniform statistical inference on gradient flows arising from empirical risk minimization. We prove a uniform central limit theorem that characterizes the deviation between empirical and population gradient flows as a continuous-time Gaussian process over the entire nonnegative real line. Building on this result, we introduce an algorithm-aware covariance estimator that evolves jointly with the gradient flow and avoids matrix inversion, resampling, or sample splitting. We show that the covariance estimator is uniformly consistent over time and use it to construct confidence intervals for the target parameter with asymptotically valid coverage. Our results connect optimization dynamics with statistical inference and provide practical tools for uncertainty quantification in gradient-based methods.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 46 of its 78 references that are in the map. First posted on arXiv 31 May 2026.
Builds on
46 of 78 references.
References · 46
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
- 2018On Lazy Training in Differentiable Programmingcited by 675
- 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
- 2014Statistical guarantees for the EM algorithm: From population to sample-based analysiscited by 133
- 2016The Landscape of Empirical Risk for Non-convex Lossescited by 86
- 2018Robust Estimation via Robust Gradient Estimationcited by 76
- 2016Statistical Inference for Model Parameters in Stochastic Gradient Descentcited by 71
- 2022High-dimensional limit theorems for SGD: Effective dynamics and critical scalingcited by 70
- 2018Singularity, Misspecification, and the Convergence Rate of EMcited by 38
- 2020Online Covariance Matrix Estimation in Stochastic Gradient Descentcited by 37
- 2022Symmetries, flat minima, and the conserved quantities of gradient flowcited by 35
- 2015Statistical consistency and asymptotic normality for high-dimensional robust M-estimatorscited by 27
34 more
- 2021Differentially private inference via noisy optimizationcited by 27
- 2021Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methodscited by 14
- 2025Gaussian Approximation and Multiplier Bootstrap for Stochastic Gradient Descentcited by 10
- 2019Gradient flows and proximal splitting methods: A unified view on accelerated and stochastic optimizationcited by 9
- 2025Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularizationcited by 8
- 2023Statistical Inference with Stochastic Gradient Methods under φ-mixing Datacited by 7
- 2026High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networkscited by 7
- 2019Path Length Bounds for Gradient Descent and Flowcited by 6
- 2026SGD with Dependent Data: Optimal Estimation, Regret, and Inferencecited by 6
- 2025Precise gradient descent training dynamics for finite-width multi-layer neural networkscited by 5
- 2023Local Conditions for Global Convergence of Gradient Flows and Proximal Point Sequences in Metric Spacescited by 4
- 2024Limit Theorems for Stochastic Gradient Descent with Infinite Variancecited by 4
- 2025Long-time dynamics and universality of nonconvex gradient descentcited by 4
- 2025Improved Central Limit Theorem and Bootstrap Approximations for Linear Stochastic Approximationcited by 4
- 2026A Researcher's Guide to Empirical Risk Minimizationcited by 4
- 2022Optimal One-pass Nonparametric Estimation Under Memory Constraintcited by 3
- 2024Uncertainty quantification for iterative algorithms in linear models with application to early stoppingcited by 3
- 2024Asymptotic Time-Uniform Inference for Parameters in Averaged Stochastic Approximationcited by 3
- 2026High-dimensional learning dynamics of multi-pass Stochastic Gradient Descent in multi-index modelscited by 3
- 2026Full-Batch Gradient Descent Outperforms One-Pass SGD: Sample Complexity Separation in Single-Index Learningcited by 3
- 2024Noisy phase retrieval from subgaussian measurementscited by 2
- 2024Gradient descent inference in empirical risk minimizationcited by 2
- 2025Statistical Inference for Online Algorithmscited by 2
- 2025Statistical Inference for Differentially Private Stochastic Gradient Descentcited by 2
- 2025Learning single index model with gradient descent: spectral initialization and precise asymptoticscited by 2
- 2025Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspectivecited by 2
- 2026High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theorycited by 2
- 2026High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noisecited by 2
- 2026Statistical Inference for Stochastic Gradient Descent: Beyond Finite Variancecited by 2
- 2023Statistical Inference with Limited Memory: A Surveycited by 1
- 2025State evolution beyond first-order methods I: Rigorous predictions and finite-sample guaranteescited by 1
- 2025Statistical Inference for Linear Functionals of Online Least-squares SGD when t gtrsim d1+δcited by 1
- 2026Two-Stage Robust Sparse Gradient Methods for Regression Under Heavy-Tailed Designscited by 1
- 2026Functional Central Limit Theorem for Stochastic Gradient Descentcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.