{"total":66,"rows":[{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2210.14199","title":"Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models","created":"2022-10-25","cited_by_count":84,"area":null},{"id":"2307.12306","title":"Tackling the Curse of Dimensionality with Physics-Informed Neural Networks","created":"2023-07-23","cited_by_count":79,"area":null},{"id":"2211.05729","title":"How Does Sharpness-Aware Minimization Minimize Sharpness?","created":"2022-11-10","cited_by_count":65,"area":null},{"id":"2105.01650","title":"Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis","created":"2021-05-04","cited_by_count":39,"area":null},{"id":"2307.11007","title":"Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization","created":"2023-07-20","cited_by_count":39,"area":null},{"id":"2102.09385","title":"Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes","created":"2021-02-16","cited_by_count":28,"area":null},{"id":"2102.09924","title":"A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions","created":"2021-02-19","cited_by_count":14,"area":null},{"id":"2405.20763","title":"Improving Generalization and Convergence by Enhancing Implicit Regularization","created":"2024-05-31","cited_by_count":14,"area":null},{"id":"2408.09966","title":"Mask in the Mirror: Implicit Sparsification","created":"2024-08-19","cited_by_count":14,"area":null},{"id":"2108.05643","title":"On minimal representations of shallow ReLU networks","created":"2021-08-12","cited_by_count":13,"area":null},{"id":"2111.06328","title":"Stationary Behavior of Constant Stepsize SGD Type Algorithms: An Asymptotic Characterization","created":"2021-11-11","cited_by_count":13,"area":null},{"id":"2103.10922","title":"Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions","created":"2021-03-19","cited_by_count":11,"area":null},{"id":"2107.04479","title":"Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation","created":"2021-07-09","cited_by_count":11,"area":null},{"id":"2409.08770","title":"Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent","created":"2024-09-13","cited_by_count":11,"area":null},{"id":"2108.04620","title":"A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions","created":"2021-08-10","cited_by_count":10,"area":null},{"id":"2104.00277","title":"A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions","created":"2021-04-01","cited_by_count":9,"area":null},{"id":"2108.08106","title":"Existence, uniqueness, and convergence rates for gradient flows in the training of artificial neural networks with ReLU activation","created":"2021-08-18","cited_by_count":9,"area":null},{"id":"2112.07369","title":"Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions","created":"2021-12-13","cited_by_count":9,"area":null},{"id":"2402.05155","title":"Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks","created":"2024-02-07","cited_by_count":9,"area":null},{"id":"2201.11989","title":"Existence and Estimation of Critical Batch Size for Training Generative Adversarial Networks with Two Time-Scale Update Rule","created":"2022-01-28","cited_by_count":8,"area":null},{"id":"2302.03550","title":"Exponential convergence rates for momentum stochastic gradient descent in the overparametrized setting","created":"2023-02-07","cited_by_count":8,"area":null},{"id":"2407.20209","title":"Characterizing Dynamical Stability of Stochastic Gradient Descent in Overparameterized Learning","created":"2024-07-29","cited_by_count":8,"area":null},{"id":"2404.12293","title":"Singular-limit analysis of gradient descent with noise injection","created":"2024-04-18","cited_by_count":7,"area":null},{"id":"2402.15344","title":"Iteration and Stochastic First-order Oracle Complexities of Stochastic Gradient Descent using Constant and Decaying Learning Rates","created":"2024-02-23","cited_by_count":5,"area":null},{"id":"2405.13592","title":"Almost sure convergence rates of stochastic gradient methods under gradient domination","created":"2024-05-22","cited_by_count":5,"area":"deeprl"},{"id":"2206.13290","title":"Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness","created":"2022-06-27","cited_by_count":4,"area":null},{"id":"2208.12401","title":"Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation","created":"2022-08-26","cited_by_count":4,"area":null},{"id":"2104.02372","title":"The Fragility of Noise Estimation in Kalman Filter: Optimization Can Handle Model-Misspecification","created":"2021-04-06","cited_by_count":3,"area":null},{"id":"2304.09221","title":"Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks","created":"2023-04-18","cited_by_count":3,"area":null},{"id":"2305.19685","title":"Deep Stochastic Mechanics","created":"2023-05-31","cited_by_count":3,"area":"diffusion"},{"id":"2306.09778","title":"Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent","created":"2023-06-16","cited_by_count":3,"area":null},{"id":"2312.00462","title":"Learning Unorthogonalized Matrices for Rotation Estimation","created":"2023-12-01","cited_by_count":3,"area":null},{"id":"2403.13245","title":"Federated reinforcement learning for robot motion planning with zero-shot generalization","created":"2024-03-20","cited_by_count":3,"area":"deeprl"},{"id":"2404.18780","title":"Optimal time sampling in physics-informed neural networks","created":"2024-04-29","cited_by_count":3,"area":null},{"id":"2410.16849","title":"Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality","created":"2024-10-22","cited_by_count":3,"area":null},{"id":"2507.05164","title":"A Dynamical Systems Perspective on the Analysis of Neural Networks","created":"2025-07-07","cited_by_count":3,"area":null},{"id":"2101.07354","title":"Exact Recovery of Community Structures Using DeepWalk and Node2vec","created":"2021-01-18","cited_by_count":2,"area":null},{"id":"2302.09815","title":"On the Stability and Generalization of Triplet Learning","created":"2023-02-20","cited_by_count":2,"area":null},{"id":"2310.12969","title":"Demystifying the Myths and Legends of Nonconvex Convergence of SGD","created":"2023-10-19","cited_by_count":2,"area":null},{"id":"2311.08745","title":"Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization","created":"2023-11-15","cited_by_count":2,"area":null},{"id":"2402.03467","title":"Stochastic Modified Flows for Riemannian Stochastic Gradient Descent","created":"2024-02-02","cited_by_count":2,"area":null},{"id":"2402.06388","title":"Convergence of a L2 regularized Policy Gradient Algorithm for the Multi Armed Bandit","created":"2024-02-09","cited_by_count":2,"area":null},{"id":"2512.13853","title":"Dropout Neural Network Training Viewed from a Percolation Perspective","created":"2025-12-15","cited_by_count":2,"area":null},{"id":"2108.11713","title":"The Number of Steps Needed for Nonconvex Optimization of a Deep Learning Optimizer is a Rational Function of Batch Size","created":"2021-08-26","cited_by_count":1,"area":null},{"id":"2208.09814","title":"Critical Bach Size Minimizes Stochastic First-Order Oracle Complexity of Deep Learning Optimizer using Hyperparameters Close to One","created":"2022-08-21","cited_by_count":1,"area":null},{"id":"2210.16196","title":"Convergence analysis of a quasi-Monte Carlo-based deep learning algorithm for solving partial differential equations","created":"2022-10-28","cited_by_count":1,"area":null},{"id":"2312.10490","title":"Spatial Deep Learning for Site-Specific Movement Optimization of Aerial Base Stations","created":"2023-12-16","cited_by_count":1,"area":null},{"id":"2401.08327","title":"Learn What You Need in Personalized Federated Learning","created":"2024-01-16","cited_by_count":1,"area":null},{"id":"2409.00908","title":"EnsLoss: Stochastic Calibrated Loss Ensembles for Preventing Overfitting in Classification","created":"2024-09-02","cited_by_count":1,"area":null},{"id":"2409.09984","title":"Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate","created":"2024-09-16","cited_by_count":1,"area":null},{"id":"2412.11400","title":"Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks","created":"2024-12-16","cited_by_count":1,"area":null},{"id":"2509.17815","title":"Global Optimization via Softmin Energy Minimization","created":"2025-09-22","cited_by_count":1,"area":null},{"id":"2105.14694","title":"Combining resampling and reweighting for faithful stochastic optimization","created":"2021-05-31","cited_by_count":0,"area":null},{"id":"2112.07163","title":"Minimization of Stochastic First-order Oracle Complexity of Adaptive Methods for Nonconvex Optimization","created":"2021-12-14","cited_by_count":0,"area":null},{"id":"2203.10973","title":"A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima","created":"2022-03-21","cited_by_count":0,"area":null},{"id":"2209.05372","title":"Convergence of Batch Updating Methods with Approximate Gradients and/or Noisy Measurements: Theory and Computational Results","created":"2022-09-12","cited_by_count":0,"area":null},{"id":"2307.13831","title":"Relationship between Batch Size and Number of Steps Needed for Nonconvex Optimization of Stochastic Gradient Descent using Armijo Line Search","created":"2023-07-25","cited_by_count":0,"area":null},{"id":"2310.01770","title":"A simple connection from loss flatness to compressed neural representations","created":"2023-10-03","cited_by_count":0,"area":null},{"id":"2507.04779","title":"Constructive Universal Approximation and Sure Convergence for Multi-Layer Neural Networks","created":"2025-07-07","cited_by_count":0,"area":null},{"id":"2512.10188","title":"The Interplay of Statistics and Noisy Optimization: Learning Linear Predictors with Random Data Weights","created":"2025-12-11","cited_by_count":0,"area":null},{"id":"2604.00241","title":"Softmax gradient policy for variance minimization and risk-averse multi armed bandits","created":"2026-03-31","cited_by_count":0,"area":null},{"id":"2605.03752","title":"Vanishing L2 regularization for the softmax Multi Armed Bandit","created":"2026-05-05","cited_by_count":0,"area":null},{"id":"2605.14663","title":"Optimal Asymptotic Rates for (Stochastic) Gradient Descent under the Local PL-Condition: A Geometric Approach","created":"2026-05-14","cited_by_count":0,"area":null},{"id":"2606.22150","title":"Parameterized Representations via Implicit Stochastic Modulation for High-Dimensional and High-Order Neural PDE Solvers","created":"2026-06-20","cited_by_count":0,"area":null}],"next_cursor":null}