{"list":"other","total":79,"rows":[{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1902.06720","title":"Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent","created":"2019-02-18","cited_by_count":738},{"id":"2008.08186","title":"Prevalence of Neural Collapse during the terminal phase of deep learning training","created":"2020-08-18","cited_by_count":719},{"id":"1812.07956","title":"On Lazy Training in Differentiable Programming","created":"2018-12-19","cited_by_count":675},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1901.10159","title":"An Investigation into Neural Net Optimization via Hessian Eigenvalue Density","created":"2019-01-29","cited_by_count":292},{"id":"1912.07145","title":"PyHessian: Neural Networks Through the Lens of the Hessian","created":"2019-12-16","cited_by_count":292},{"id":"1706.04454","title":"Empirical Analysis of the Hessian of Over-Parametrized Neural Networks","created":"2017-06-14","cited_by_count":290},{"id":"2303.14186","title":"TRAK: Attributing Model Behavior at Scale","created":"2023-03-24","cited_by_count":283},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"2010.15110","title":"Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel","created":"2020-10-28","cited_by_count":187},{"id":"2405.18392","title":"Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations","created":"2024-05-28","cited_by_count":170},{"id":"2405.15682","title":"The Road Less Scheduled","created":"2024-05-24","cited_by_count":163},{"id":"1611.07476","title":"Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond","created":"2016-11-22","cited_by_count":159},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155},{"id":"1712.06559","title":"The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning","created":"2017-12-18","cited_by_count":150},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"1511.06251","title":"Stochastic modified equations and adaptive stochastic gradient algorithms","created":"2015-11-19","cited_by_count":142},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124},{"id":"2210.05643","title":"A Kernel-Based View of Language Model Fine-Tuning","created":"2022-10-11","cited_by_count":121},{"id":"1907.04164","title":"Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model","created":"2019-07-09","cited_by_count":117},{"id":"2111.00034","title":"Neural Networks as Kernel Learners: The Silent Alignment Effect","created":"2021-10-29","cited_by_count":106},{"id":"2205.10287","title":"On the SDEs and Scaling Rules for Adaptive Gradient Algorithms","created":"2022-05-20","cited_by_count":104},{"id":"2207.14484","title":"Adaptive Gradient Methods at the Edge of Stability","created":"2022-07-29","cited_by_count":101},{"id":"2008.11865","title":"Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra","created":"2020-08-27","cited_by_count":94},{"id":"1707.06386","title":"Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains","created":"2017-07-20","cited_by_count":88},{"id":"1807.05031","title":"On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length","created":"2018-07-13","cited_by_count":88},{"id":"2309.04827","title":"Neurons in Large Language Models: Dead, N-gram, Positional","created":"2023-09-09","cited_by_count":88},{"id":"2204.01050","title":"Understanding the unstable convergence of gradient descent","created":"2022-04-03","cited_by_count":86},{"id":"1910.01619","title":"Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks","created":"2019-10-03","cited_by_count":79},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73},{"id":"2103.12692","title":"Benign Overfitting of Constant-Stepsize SGD for Linear Regression","created":"2021-03-23","cited_by_count":71},{"id":"1901.08244","title":"Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians","created":"2019-01-24","cited_by_count":68},{"id":"2405.15074","title":"4+3 Phases of Compute-Optimal Neural Scaling Laws","created":"2024-05-23","cited_by_count":68},{"id":"2207.02628","title":"The alignment property of SGD noise and how it helps select flat minima: A stability analysis","created":"2022-07-06","cited_by_count":64},{"id":"2410.24206","title":"Understanding Optimization in Deep Learning with Central Flows","created":"2024-10-31","cited_by_count":64},{"id":"2105.13462","title":"On Linear Stability of SGD and Input-Smoothness of Neural Networks","created":"2021-05-27","cited_by_count":63},{"id":"2006.09092","title":"Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training","created":"2020-06-16","cited_by_count":50},{"id":"1803.05591","title":"On the insufficiency of existing momentum schemes for Stochastic Optimization","created":"2018-03-15","cited_by_count":49},{"id":"2106.06770","title":"What can linearized neural networks actually say about generalization?","created":"2021-06-12","cited_by_count":49},{"id":"2310.07831","title":"Optimal Linear Decay Learning Rate Schedules and Further Refinements","created":"2023-10-11","cited_by_count":43},{"id":"2102.03183","title":"Last iterate convergence of SGD for Least-Squares in the Interpolation regime","created":"2021-02-05","cited_by_count":42},{"id":"2206.10012","title":"Limitations of the NTK for Understanding Generalization in Deep Learning","created":"2022-06-20","cited_by_count":35},{"id":"2501.18965","title":"The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training","created":"2025-01-31","cited_by_count":33},{"id":"2412.20553","title":"Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD","created":"2024-12-29","cited_by_count":30},{"id":"2105.10585","title":"Properties of the After Kernel","created":"2021-05-21","cited_by_count":27},{"id":"2411.15958","title":"Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise","created":"2024-11-24","cited_by_count":23},{"id":"2301.08203","title":"An SDE for Modeling SAM: Theory and Insights","created":"2023-01-19","cited_by_count":22},{"id":"2311.02076","title":"Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos","created":"2023-11-03","cited_by_count":22},{"id":"2105.06595","title":"Analysis of stochastic Lanczos quadrature for spectrum approximation","created":"2021-05-13","cited_by_count":17},{"id":"2510.09378","title":"The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton","created":"2025-10-10","cited_by_count":17},{"id":"1802.03451","title":"Estimating the Spectral Density of Large Implicit Matrices","created":"2018-02-09","cited_by_count":16},{"id":"2106.03696","title":"Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models","created":"2021-06-07","cited_by_count":16},{"id":"2505.16098","title":"Dimension-adapted Momentum Outscales SGD","created":"2025-05-22","cited_by_count":12},{"id":"2002.04010","title":"Taylorized Training: Towards Better Approximation of Neural Network Training at Finite Width","created":"2020-02-10","cited_by_count":11},{"id":"2206.11124","title":"A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta","created":"2022-06-22","cited_by_count":11},{"id":"2306.07850","title":"Exact Mean Square Linear Stability Analysis for SGD","created":"2023-06-13","cited_by_count":11},{"id":"2602.03702","title":"Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging","created":"2026-02-03","cited_by_count":10},{"id":"2603.15958","title":"Deriving Hyperparameter Scaling Laws via Modern Optimization Theory","created":"2026-03-16","cited_by_count":10},{"id":"2604.14108","title":"Momentum Further Constrains Sharpness at the Edge of Stochastic Stability","created":"2026-04-15","cited_by_count":10},{"id":"1712.06038","title":"The proximal point method revisited","created":"2017-12-17","cited_by_count":9},{"id":"2203.01744","title":"Accelerated SGD for Non-Strongly-Convex Least Squares","created":"2022-03-03","cited_by_count":9},{"id":"2409.09745","title":"The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization","created":"2024-09-15","cited_by_count":8},{"id":"2509.17251","title":"Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularization","created":"2025-09-21","cited_by_count":8},{"id":"2605.19095","title":"ScheduleFree+: Scaling Learning-Rate-Free & Schedule-Free Learning to Large Language Models","created":"2026-05-18","cited_by_count":7},{"id":"2507.12837","title":"Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability","created":"2025-07-17","cited_by_count":6},{"id":"2510.14717","title":"Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling","created":"2025-10-16","cited_by_count":6},{"id":"2605.07870","title":"Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer","created":"2026-05-08","cited_by_count":6},{"id":"1909.12108","title":"GradVis: Visualization and Second Order Analysis of Optimization Surfaces during the Training of Deep Neural Networks","created":"2019-09-26","cited_by_count":5},{"id":"2505.11564","title":"HessFormer: Hessians at Foundation Scale","created":"2025-05-16","cited_by_count":4},{"id":"2506.00181","title":"On the Interaction of Batch Noise, Adaptivity, and Compression, under $(L_0,L_1)$-Smoothness: An SDE Approach","created":"2025-05-30","cited_by_count":4},{"id":"2602.19241","title":"Scaling Laws for Precision in High-Dimensional Linear Regression","created":"2026-02-22","cited_by_count":4},{"id":"2606.19179","title":"Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods","created":"2026-06-17","cited_by_count":2},{"id":"2602.00816","title":"Hessian Spectral Analysis at Foundation Model Scale","created":"2026-01-31","cited_by_count":1},{"id":"2606.14259","title":"Beyond a Single Explanation of the Adam--SGD Gap","created":"2026-06-12","cited_by_count":1}],"next_cursor":null}