{"total":69,"rows":[{"id":"2305.17333","title":"Fine-Tuning Language Models with Just Forward Passes","created":"2023-05-27","cited_by_count":345,"area":null},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107,"area":null},{"id":"2205.10287","title":"On the SDEs and Scaling Rules for Adaptive Gradient Algorithms","created":"2022-05-20","cited_by_count":104,"area":null},{"id":"2206.04041","title":"Neural Collapse: A Review on Modelling Principles and Generalization","created":"2022-06-08","cited_by_count":95,"area":null},{"id":"2211.13609","title":"PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization","created":"2022-11-24","cited_by_count":76,"area":null},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73,"area":null},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72,"area":null},{"id":"2206.04030","title":"High-dimensional limit theorems for SGD: Effective dynamics and critical scaling","created":"2022-06-08","cited_by_count":70,"area":null},{"id":"2207.02628","title":"The alignment property of SGD noise and how it helps select flat minima: A stability analysis","created":"2022-07-06","cited_by_count":64,"area":null},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63,"area":null},{"id":"2305.17212","title":"Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks","created":"2023-05-26","cited_by_count":53,"area":null},{"id":"2105.02716","title":"Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks","created":"2021-05-06","cited_by_count":50,"area":null},{"id":"2406.18533","title":"On Scaling Up 3D Gaussian Splatting Training","created":"2024-06-26","cited_by_count":49,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2307.13813","title":"How to Scale Your EMA","created":"2023-07-25","cited_by_count":24,"area":null},{"id":"2106.04881","title":"Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms","created":"2021-06-09","cited_by_count":23,"area":null},{"id":"2505.23971","title":"Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training","created":"2025-05-29","cited_by_count":23,"area":null},{"id":"2301.08203","title":"An SDE for Modeling SAM: Theory and Insights","created":"2023-01-19","cited_by_count":22,"area":null},{"id":"2410.23922","title":"Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training","created":"2024-10-31","cited_by_count":22,"area":null},{"id":"2501.12666","title":"Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization","created":"2025-01-22","cited_by_count":21,"area":null},{"id":"2107.09133","title":"The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion","created":"2021-07-19","cited_by_count":19,"area":null},{"id":"2402.07193","title":"Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent","created":"2024-02-11","cited_by_count":19,"area":null},{"id":"2307.15196","title":"The Marginal Value of Momentum for Small Learning Rate SGD","created":"2023-07-27","cited_by_count":18,"area":null},{"id":"2201.13011","title":"On the Power-Law Hessian Spectrums in Deep Learning","created":"2022-01-31","cited_by_count":16,"area":null},{"id":"2306.02913","title":"Decentralized SGD and Average-direction SAM are Asymptotically Equivalent","created":"2023-06-05","cited_by_count":15,"area":null},{"id":"2212.02083","title":"On the Overlooked Structure of Stochastic Gradients","created":"2022-12-05","cited_by_count":13,"area":null},{"id":"2309.10688","title":"On the different regimes of Stochastic Gradient Descent","created":"2023-09-19","cited_by_count":12,"area":null},{"id":"2211.10691","title":"Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States","created":"2022-11-19","cited_by_count":11,"area":null},{"id":"2302.09693","title":"mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization","created":"2023-02-19","cited_by_count":10,"area":null},{"id":"2209.14501","title":"On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks","created":"2022-09-29","cited_by_count":9,"area":null},{"id":"2310.00692","title":"A Theoretical Analysis of Noise Geometry in Stochastic Gradient Descent","created":"2023-10-01","cited_by_count":9,"area":null},{"id":"2107.11774","title":"SGD with a Constant Large Learning Rate Can Converge to Local Maxima","created":"2021-07-25","cited_by_count":8,"area":null},{"id":"2110.05960","title":"Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations","created":"2021-10-11","cited_by_count":8,"area":null},{"id":"2406.11110","title":"How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD","created":"2024-06-17","cited_by_count":8,"area":null},{"id":"2302.07125","title":"Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent","created":"2023-02-14","cited_by_count":7,"area":null},{"id":"2308.06671","title":"Noise Balance and Stationary Distribution of Stochastic Gradient Descent","created":"2023-08-13","cited_by_count":7,"area":null},{"id":"2309.03964","title":"REALM: Robust Entropy Adaptive Loss Minimization for Improved Single-Sample Test-Time Adaptation","created":"2023-09-07","cited_by_count":7,"area":null},{"id":"2401.11600","title":"Understanding the Generalization Benefits of Late Learning Rate Decay","created":"2024-01-21","cited_by_count":6,"area":null},{"id":"2402.12508","title":"SDEs for Minimax Optimization","created":"2024-02-19","cited_by_count":6,"area":null},{"id":"2510.14717","title":"Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling","created":"2025-10-16","cited_by_count":6,"area":null},{"id":"2205.13799","title":"Generalization Bounds for Gradient Methods via Discrete and Continuous Prior","created":"2022-05-27","cited_by_count":5,"area":null},{"id":"2210.11452","title":"Global Convergence of SGD On Two Layer Neural Nets","created":"2022-10-20","cited_by_count":5,"area":null},{"id":"2310.14423","title":"A Quadratic Synchronization Rule for Distributed Deep Learning","created":"2023-10-22","cited_by_count":5,"area":null},{"id":"2502.17009","title":"Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs","created":"2025-02-24","cited_by_count":5,"area":null},{"id":"2201.13195","title":"Memory-Efficient Backpropagation through Large Linear Layers","created":"2022-01-31","cited_by_count":4,"area":null},{"id":"2406.03768","title":"Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective","created":"2024-06-06","cited_by_count":4,"area":null},{"id":"2309.05337","title":"Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems","created":"2023-09-11","cited_by_count":3,"area":null},{"id":"2407.02322","title":"Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descent","created":"2024-07-02","cited_by_count":3,"area":null},{"id":"2402.03467","title":"Stochastic Modified Flows for Riemannian Stochastic Gradient Descent","created":"2024-02-02","cited_by_count":2,"area":null},{"id":"2408.14677","title":"Can Optimization Trajectories Explain Multi-Task Transfer?","created":"2024-08-26","cited_by_count":2,"area":null},{"id":"2502.07584","title":"Generalization Bounds for Markov Algorithms through Entropy Flow Computations","created":"2025-02-11","cited_by_count":2,"area":null},{"id":"2509.18001","title":"Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise","created":"2025-09-22","cited_by_count":2,"area":null},{"id":"2602.05600","title":"On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2603.03226","title":"Adaptive Methods Are Preferable in High Privacy Settings: An SDE Perspective","created":"2026-03-03","cited_by_count":2,"area":null},{"id":"2210.11257","title":"A note on diffusion limits for stochastic gradient descent","created":"2022-10-20","cited_by_count":1,"area":null},{"id":"2305.13141","title":"Tight conditions for when the NTK approximation is valid","created":"2023-05-22","cited_by_count":1,"area":null},{"id":"2311.14120","title":"Weight fluctuations in (deep) linear neural networks and a derivation of the inverse-variance flatness relation","created":"2023-11-23","cited_by_count":1,"area":null},{"id":"2406.11159","title":"Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework","created":"2024-06-17","cited_by_count":1,"area":null},{"id":"2412.02153","title":"Revisiting the Initial Steps in Adaptive Gradient Descent Optimization","created":"2024-12-03","cited_by_count":1,"area":null},{"id":"2507.10623","title":"Flows and Diffusions on the Neural Manifold","created":"2025-07-14","cited_by_count":1,"area":null},{"id":"2511.02258","title":"Limit Theorems for Stochastic Gradient Descent in High-Dimensional Single-Layer Networks","created":"2025-11-04","cited_by_count":1,"area":null},{"id":"2602.05393","title":"Late-to-Early Training: LET LLMs Learn Earlier, So Faster and Better","created":"2026-02-05","cited_by_count":1,"area":null},{"id":"2607.23012","title":"Mini-batch Noise Lowers Sharpness via Dominant-Subspace Fluctuations","created":"2026-07-25","cited_by_count":1,"area":null},{"id":"2305.18864","title":"Stochastic Gradient Langevin Dynamics Based on Quantization with Increasing Resolution","created":"2023-05-30","cited_by_count":0,"area":null},{"id":"2406.03696","title":"Error dynamics of mini-batch gradient descent with random reshuffling for least squares regression","created":"2024-06-06","cited_by_count":0,"area":null},{"id":"2410.19371","title":"Noise-Aware Differentially Private Variational Inference","created":"2024-10-25","cited_by_count":0,"area":null},{"id":"2504.14882","title":"Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness","created":"2025-04-21","cited_by_count":0,"area":null},{"id":"2601.19730","title":"Stability and Generalization of Stochastic Nonconvex Optimization under Heavy-Tailed Noise: From Gradient Norm to Goldstein Stationarity","created":"2026-01-27","cited_by_count":0,"area":null},{"id":"2606.18183","title":"A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise","created":"2026-06-16","cited_by_count":0,"area":null}],"next_cursor":null}