{"total":68,"rows":[{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427,"area":null},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137,"area":null},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2206.04030","title":"High-dimensional limit theorems for SGD: Effective dynamics and critical scaling","created":"2022-06-08","cited_by_count":70,"area":null},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63,"area":null},{"id":"2406.17748","title":"A New Perspective on Shampoo's Preconditioner","created":"2024-06-25","cited_by_count":59,"area":null},{"id":"2106.16225","title":"Analytic Insights into Structure and Rank of Neural Network Hessian Maps","created":"2021-06-30","cited_by_count":52,"area":null},{"id":"2405.16002","title":"Does SGD really happen in tiny subspaces?","created":"2024-05-25","cited_by_count":36,"area":null},{"id":"2206.02659","title":"Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees","created":"2022-06-06","cited_by_count":34,"area":null},{"id":"2103.01519","title":"Hessian Eigenspectra of More Realistic Nonlinear Models","created":"2021-03-02","cited_by_count":33,"area":null},{"id":"2306.04815","title":"Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning","created":"2023-06-07","cited_by_count":31,"area":null},{"id":"2105.09557","title":"Power-law escape rate of SGD","created":"2021-05-20","cited_by_count":29,"area":null},{"id":"2107.05802","title":"How many degrees of freedom do we need to train deep networks: a loss landscape perspective","created":"2021-07-13","cited_by_count":28,"area":null},{"id":"2207.05952","title":"Implicit regularization of dropout","created":"2022-07-13","cited_by_count":25,"area":null},{"id":"2308.02760","title":"Neural Collapse in the Intermediate Hidden Layers of Classification Neural Networks","created":"2023-08-05","cited_by_count":21,"area":null},{"id":"2405.15603","title":"Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks","created":"2024-05-24","cited_by_count":21,"area":null},{"id":"2305.18512","title":"A Rainbow in Deep Network Black Boxes","created":"2023-05-29","cited_by_count":18,"area":null},{"id":"2106.02624","title":"ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure","created":"2021-06-04","cited_by_count":16,"area":null},{"id":"2201.13011","title":"On the Power-Law Hessian Spectrums in Deep Learning","created":"2022-01-31","cited_by_count":16,"area":null},{"id":"2110.14163","title":"Does the Data Induce Capacity Control in Deep Learning?","created":"2021-10-27","cited_by_count":14,"area":null},{"id":"2306.05651","title":"Differentially Private Sharpness-Aware Training","created":"2023-06-09","cited_by_count":14,"area":null},{"id":"2607.21716","title":"A Defense of the Quadratic Model","created":"2026-07-23","cited_by_count":14,"area":null},{"id":"2212.02083","title":"On the Overlooked Structure of Stochastic Gradients","created":"2022-12-05","cited_by_count":13,"area":null},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13,"area":null},{"id":"2410.23169","title":"The Persistence of Neural Collapse Despite Low-Rank Bias","created":"2024-10-30","cited_by_count":13,"area":null},{"id":"2505.02809","title":"Towards Quantifying the Hessian Structure of Neural Networks","created":"2025-05-05","cited_by_count":13,"area":null},{"id":"2111.13331","title":"Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping","created":"2021-11-26","cited_by_count":12,"area":null},{"id":"2305.15850","title":"Stochastic Modified Equations and Dynamics of Dropout Algorithm","created":"2023-05-25","cited_by_count":11,"area":null},{"id":"2404.06106","title":"Unifying Low Dimensional Spectra in Deep Learning","created":"2024-04-09","cited_by_count":11,"area":null},{"id":"2501.19183","title":"Position: Curvature Matrices Should Be Democratized via Linear Operators","created":"2025-01-31","cited_by_count":9,"area":null},{"id":"2102.06740","title":"Appearance of Random Matrix Theory in Deep Learning","created":"2021-02-12","cited_by_count":7,"area":null},{"id":"2305.14683","title":"On progressive sharpening, flat minima and generalisation","created":"2023-05-24","cited_by_count":7,"area":null},{"id":"2306.07104","title":"Unveiling the Hessian's Connection to the Decision Boundary","created":"2023-06-12","cited_by_count":5,"area":null},{"id":"2504.15208","title":"Compute-Optimal LLMs Provably Generalize Better With Scale","created":"2025-04-21","cited_by_count":5,"area":null},{"id":"2605.05683","title":"Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimization","created":"2026-05-07","cited_by_count":5,"area":null},{"id":"2402.03579","title":"Deconstructing the Goldilocks Zone of Neural Network Initialization","created":"2024-02-05","cited_by_count":4,"area":null},{"id":"2605.22432","title":"AMUSE: Anytime Muon with Stable Gradient Evaluation","created":"2026-05-21","cited_by_count":4,"area":null},{"id":"2607.16821","title":"First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers","created":"2026-07-18","cited_by_count":4,"area":"mechinterp"},{"id":"2307.11948","title":"The instabilities of large learning rate training: a loss landscape view","created":"2023-07-22","cited_by_count":3,"area":null},{"id":"2105.03788","title":"Dynamic Game Theoretic Neural Optimizer","created":"2021-05-08","cited_by_count":2,"area":null},{"id":"2111.01022","title":"Dropout in Training Neural Networks: Flatness of Solution and Noise Structure","created":"2021-11-01","cited_by_count":2,"area":null},{"id":"2507.03545","title":"DOME: Improving Signal-to-Noise in Stochastic Gradient Descent via Sharp-Direction Subspace Filtering","created":"2025-07-04","cited_by_count":2,"area":null},{"id":"2602.05600","title":"On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2605.08237","title":"Distributional Spectral Diagnostics for Localizing Grokking Transitions","created":"2026-05-07","cited_by_count":2,"area":null},{"id":"2107.05328","title":"Structured Directional Pruning via Perturbation Orthogonal Projection","created":"2021-07-12","cited_by_count":1,"area":null},{"id":"2201.06964","title":"There is a Singularity in the Loss Landscape","created":"2022-01-12","cited_by_count":1,"area":null},{"id":"2302.08484","title":"FOSI: Hybrid First and Second Order Optimization","created":"2023-02-16","cited_by_count":1,"area":null},{"id":"2406.00127","title":"Training on the Edge of Stability Is Caused by Layerwise Jacobian Alignment","created":"2024-05-31","cited_by_count":1,"area":null},{"id":"2409.11995","title":"Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes","created":"2024-09-18","cited_by_count":1,"area":null},{"id":"2504.14701","title":"Connecting Parameter Magnitudes and Hessian Eigenspaces at Scale using Sketched Methods","created":"2025-04-20","cited_by_count":1,"area":null},{"id":"2505.11972","title":"Accelerating Neural Network Training Along Sharp and Flat Directions","created":"2025-05-17","cited_by_count":1,"area":null},{"id":"2607.23012","title":"Mini-batch Noise Lowers Sharpness via Dominant-Subspace Fluctuations","created":"2026-07-25","cited_by_count":1,"area":null},{"id":"2201.07934","title":"Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks","created":"2022-01-03","cited_by_count":0,"area":null},{"id":"2305.18490","title":"SANE: The phases of gradient descent through Sharpness Adjusted Number of Effective parameters","created":"2023-05-29","cited_by_count":0,"area":null},{"id":"2402.02711","title":"Architectural Strategies for the optimization of Physics-Informed Neural Networks","created":"2024-02-05","cited_by_count":0,"area":null},{"id":"2406.07712","title":"Loss Gradient Gaussian Width based Generalization and Optimization Guarantees","created":"2024-06-11","cited_by_count":0,"area":null},{"id":"2412.17613","title":"Can Stability be Detrimental? Better Generalization through Gradient Descent Instabilities","created":"2024-12-23","cited_by_count":0,"area":null},{"id":"2504.18017","title":"Non-identifiability distinguishes Neural Networks among Parametric Models","created":"2025-04-25","cited_by_count":0,"area":null},{"id":"2506.08428","title":"Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings","created":"2025-06-10","cited_by_count":0,"area":null},{"id":"2604.11639","title":"Inter-Layer Hessian Analysis of Neural Networks with DAG Architectures","created":"2026-04-13","cited_by_count":0,"area":null},{"id":"2605.29983","title":"Improving Adversarial Robustness of Attribution via Implicit Regularization","created":"2026-05-28","cited_by_count":0,"area":null},{"id":"2605.31324","title":"Inconsistency-Aware Minimization: Improving Generalization with Unlabeled Data","created":"2026-05-29","cited_by_count":0,"area":null},{"id":"2607.07845","title":"Explaining Near-Zero Hessian Eigenvalues Through Approximate Symmetries in Neural Networks","created":"2026-07-08","cited_by_count":0,"area":null},{"id":"2608.15901","title":"Layers Matter: Why Continual Learning Regularization Should Be Layer-Adaptive","created":"2026-08-16","cited_by_count":0,"area":null},{"id":"2608.19491","title":"Activation-Keyed Momentum: An Anisotropic Momentum Update via the Delta Rule","created":"2026-08-19","cited_by_count":0,"area":null},{"id":"2609.01034","title":"The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow","created":"2026-09-01","cited_by_count":0,"area":null},{"id":"2609.01811","title":"Reinforcement learning to choose optimizers","created":"2026-09-01","cited_by_count":0,"area":"deeprl"},{"id":"2609.31983","title":"Understanding the Subspace Stabilization of the Hessian and Gradient Covariance Matrix","created":"2026-09-25","cited_by_count":0,"area":null}],"next_cursor":null}