{"total":150,"rows":[{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265,"area":null},{"id":"2106.09647","title":"Deep Learning Through the Lens of Example Difficulty","created":"2021-06-17","cited_by_count":141,"area":null},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137,"area":null},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2206.04041","title":"Neural Collapse: A Review on Modelling Principles and Generalization","created":"2022-06-08","cited_by_count":95,"area":null},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63,"area":null},{"id":"2301.11300","title":"ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients","created":"2023-01-26","cited_by_count":58,"area":null},{"id":"2503.20762","title":"ASGO: Adaptive Structured Gradient Optimization","created":"2025-03-26","cited_by_count":57,"area":null},{"id":"2402.15173","title":"Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer","created":"2024-02-23","cited_by_count":53,"area":null},{"id":"2106.16225","title":"Analytic Insights into Structure and Rank of Neural Network Hessian Maps","created":"2021-06-30","cited_by_count":52,"area":null},{"id":"2405.16002","title":"Does SGD really happen in tiny subspaces?","created":"2024-05-25","cited_by_count":36,"area":null},{"id":"2102.04396","title":"SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality","created":"2021-02-08","cited_by_count":35,"area":null},{"id":"2107.11228","title":"Taxonomizing local versus global structure in neural network loss landscapes","created":"2021-07-23","cited_by_count":34,"area":null},{"id":"2206.02659","title":"Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees","created":"2022-06-06","cited_by_count":34,"area":null},{"id":"2103.01519","title":"Hessian Eigenspectra of More Realistic Nonlinear Models","created":"2021-03-02","cited_by_count":33,"area":null},{"id":"2107.05802","title":"How many degrees of freedom do we need to train deep networks: a loss landscape perspective","created":"2021-07-13","cited_by_count":28,"area":null},{"id":"2107.12723","title":"Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel","created":"2021-07-27","cited_by_count":26,"area":null},{"id":"2309.16932","title":"Symmetry Induces Structure and Constraint of Learning","created":"2023-09-29","cited_by_count":25,"area":null},{"id":"2206.07252","title":"Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions","created":"2022-06-15","cited_by_count":23,"area":null},{"id":"2402.17457","title":"Super Consistency of Neural Network Landscapes and Learning Rate Transfer","created":"2024-02-27","cited_by_count":23,"area":null},{"id":"2406.15244","title":"AdaGrad under Anisotropic Smoothness","created":"2024-06-21","cited_by_count":22,"area":null},{"id":"2303.12922","title":"Revisiting the Fragility of Influence Functions","created":"2023-03-22","cited_by_count":20,"area":null},{"id":"2107.09133","title":"The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion","created":"2021-07-19","cited_by_count":19,"area":null},{"id":"2306.09000","title":"When and Why Momentum Accelerates SGD:An Empirical Study","created":"2023-06-15","cited_by_count":19,"area":null},{"id":"2305.18512","title":"A Rainbow in Deep Network Black Boxes","created":"2023-05-29","cited_by_count":18,"area":null},{"id":"2302.03764","title":"Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions","created":"2023-02-07","cited_by_count":17,"area":null},{"id":"2603.09697","title":"Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning","created":"2026-03-10","cited_by_count":17,"area":null},{"id":"2101.04968","title":"Learning with Gradient Descent and Weakly Convex Losses","created":"2021-01-13","cited_by_count":16,"area":null},{"id":"2106.02624","title":"ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure","created":"2021-06-04","cited_by_count":16,"area":null},{"id":"2106.03696","title":"Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models","created":"2021-06-07","cited_by_count":16,"area":null},{"id":"2107.10370","title":"Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II","created":"2021-07-21","cited_by_count":16,"area":null},{"id":"2201.13011","title":"On the Power-Law Hessian Spectrums in Deep Learning","created":"2022-01-31","cited_by_count":16,"area":null},{"id":"2202.00280","title":"Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?","created":"2022-02-01","cited_by_count":16,"area":null},{"id":"2110.14109","title":"Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums","created":"2021-10-27","cited_by_count":15,"area":null},{"id":"2306.07158","title":"Riemannian Laplace approximations for Bayesian neural networks","created":"2023-06-12","cited_by_count":15,"area":null},{"id":"2603.20527","title":"RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization","created":"2026-03-20","cited_by_count":15,"area":null},{"id":"2210.13447","title":"Precision Machine Learning","created":"2022-10-24","cited_by_count":14,"area":null},{"id":"2607.21716","title":"A Defense of the Quadratic Model","created":"2026-07-23","cited_by_count":14,"area":null},{"id":"2103.06219","title":"Why flatness does and does not correlate with generalization for deep neural networks","created":"2021-03-10","cited_by_count":13,"area":null},{"id":"2212.02083","title":"On the Overlooked Structure of Stochastic Gradients","created":"2022-12-05","cited_by_count":13,"area":null},{"id":"2304.13871","title":"Typical and atypical solutions in non-convex neural networks with discrete and continuous weights","created":"2023-04-26","cited_by_count":13,"area":null},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13,"area":null},{"id":"2410.23169","title":"The Persistence of Neural Collapse Despite Low-Rank Bias","created":"2024-10-30","cited_by_count":13,"area":null},{"id":"2505.02809","title":"Towards Quantifying the Hessian Structure of Neural Networks","created":"2025-05-05","cited_by_count":13,"area":null},{"id":"2602.11948","title":"Insights on Muon from Simple Quadratics","created":"2026-02-12","cited_by_count":13,"area":null},{"id":"2111.13331","title":"Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping","created":"2021-11-26","cited_by_count":12,"area":null},{"id":"2203.07337","title":"Phenomenology of Double Descent in Finite-Width Neural Networks","created":"2022-03-14","cited_by_count":12,"area":null},{"id":"2404.06106","title":"Unifying Low Dimensional Spectra in Deep Learning","created":"2024-04-09","cited_by_count":11,"area":null},{"id":"2406.14095","title":"Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization","created":"2024-06-20","cited_by_count":11,"area":null},{"id":"2206.15078","title":"Laplacian Autoencoders for Learning Stochastic Representations","created":"2022-06-30","cited_by_count":10,"area":null},{"id":"2305.09088","title":"The Hessian perspective into the Nature of Convolutional Neural Networks","created":"2023-05-16","cited_by_count":10,"area":null},{"id":"2305.12201","title":"GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training","created":"2023-05-20","cited_by_count":10,"area":null},{"id":"2507.14740","title":"Better Training Data Attribution via Better Inverse Hessian-Vector Products","created":"2025-07-19","cited_by_count":10,"area":null},{"id":"2208.07951","title":"On the generalization of learning algorithms that do not converge","created":"2022-08-16","cited_by_count":9,"area":null},{"id":"2312.14567","title":"Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise","created":"2023-12-22","cited_by_count":9,"area":null},{"id":"2410.10986","title":"What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis","created":"2024-10-14","cited_by_count":9,"area":null},{"id":"2411.10696","title":"HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization","created":"2024-11-16","cited_by_count":9,"area":null},{"id":"2501.19183","title":"Position: Curvature Matrices Should Be Democratized via Linear Operators","created":"2025-01-31","cited_by_count":9,"area":null},{"id":"2501.12243","title":"FOCUS: First Order Concentrated Updating Scheme","created":"2025-01-21","cited_by_count":8,"area":null},{"id":"2602.18584","title":"GIST: Targeted Data Selection for Instruction Tuning via Coupled Optimization Geometry","created":"2026-02-20","cited_by_count":8,"area":null},{"id":"2102.03793","title":"Tilting the playing field: Dynamical loss functions for machine learning","created":"2021-02-07","cited_by_count":7,"area":null},{"id":"2102.06604","title":"Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks","created":"2021-02-12","cited_by_count":7,"area":null},{"id":"2102.06740","title":"Appearance of Random Matrix Theory in Deep Learning","created":"2021-02-12","cited_by_count":7,"area":null},{"id":"2307.07950","title":"Accelerating Distributed ML Training via Selective Synchronization","created":"2023-07-16","cited_by_count":7,"area":null},{"id":"2410.08395","title":"Nesterov acceleration in benignly non-convex landscapes","created":"2024-10-10","cited_by_count":7,"area":null},{"id":"2409.12371","title":"Communication-Efficient Federated Low-Rank Update Algorithm and its Connection to Implicit Regularization","created":"2024-09-19","cited_by_count":6,"area":null},{"id":"2210.16400","title":"Flatter, faster: scaling momentum for optimal speedup of SGD","created":"2022-10-28","cited_by_count":5,"area":null},{"id":"2306.07104","title":"Unveiling the Hessian's Connection to the Decision Boundary","created":"2023-06-12","cited_by_count":5,"area":null},{"id":"2411.02139","title":"Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks","created":"2024-11-04","cited_by_count":5,"area":null},{"id":"2505.18886","title":"KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning","created":"2025-05-24","cited_by_count":5,"area":null},{"id":"2505.23737","title":"On the Convergence Analysis of Muon","created":"2025-05-29","cited_by_count":5,"area":null},{"id":"2508.13898","title":"Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training","created":"2025-08-19","cited_by_count":5,"area":null},{"id":"2309.09240","title":"High-dimensional manifold of solutions in neural networks: insights from statistical physics","created":"2023-09-17","cited_by_count":4,"area":null},{"id":"2505.24261","title":"Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining","created":"2025-05-30","cited_by_count":4,"area":null},{"id":"2605.22432","title":"AMUSE: Anytime Muon with Stable Gradient Evaluation","created":"2026-05-21","cited_by_count":4,"area":null},{"id":"2206.00823","title":"Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules","created":"2022-06-02","cited_by_count":3,"area":null},{"id":"2307.11948","title":"The instabilities of large learning rate training: a loss landscape view","created":"2023-07-22","cited_by_count":3,"area":null},{"id":"2310.17610","title":"A qualitative difference between gradient flows of convex functions in finite- and infinite-dimensional Hilbert spaces","created":"2023-10-26","cited_by_count":3,"area":null},{"id":"2410.14570","title":"Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMs","created":"2024-10-18","cited_by_count":3,"area":null},{"id":"2412.06867","title":"Lossless Model Compression via Joint Low-Rank Factorization Optimization","created":"2024-12-09","cited_by_count":3,"area":null},{"id":"2506.07661","title":"Information-Theoretic Framework for Understanding Modern Machine-Learning","created":"2025-06-09","cited_by_count":3,"area":null},{"id":"2601.21929","title":"LoRIF: Low-Rank Influence Functions for Scalable Training Data Attribution","created":"2026-01-29","cited_by_count":3,"area":null},{"id":"2602.07145","title":"Convex Dominance in Deep Learning I: A Scaling Law of Loss and Learning Rate","created":"2026-02-06","cited_by_count":3,"area":null},{"id":"2206.05810","title":"Analysis of Branch Specialization and its Application in Image Decomposition","created":"2022-06-12","cited_by_count":2,"area":null},{"id":"2207.01847","title":"PoF: Post-Training of Feature Extractor for Improving Generalization","created":"2022-07-05","cited_by_count":2,"area":null},{"id":"2210.13599","title":"Noise Injection as a Probe of Deep Learning Dynamics","created":"2022-10-24","cited_by_count":2,"area":null},{"id":"2312.16819","title":"Hidden Minima in Two-Layer ReLU Networks","created":"2023-12-28","cited_by_count":2,"area":null},{"id":"2405.01731","title":"Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization","created":"2024-05-02","cited_by_count":2,"area":null},{"id":"2408.14445","title":"Symmetry & Critical Points","created":"2024-08-26","cited_by_count":2,"area":null},{"id":"2502.02345","title":"Low Rank Based Subspace Inference for the Laplace Approximation of Bayesian Neural Networks","created":"2025-02-04","cited_by_count":2,"area":null},{"id":"2503.22478","title":"Almost Bayesian: The Fractal Dynamics of Stochastic Gradient Descent","created":"2025-03-28","cited_by_count":2,"area":null},{"id":"2507.03545","title":"DOME: Improving Signal-to-Noise in Stochastic Gradient Descent via Sharp-Direction Subspace Filtering","created":"2025-07-04","cited_by_count":2,"area":null},{"id":"2509.15651","title":"Toward Efficient Influence Function: Dropout as a Compression Tool","created":"2025-09-19","cited_by_count":2,"area":null},{"id":"2510.13680","title":"Adam or Gauss-Newton? A Comparative Study In Terms of Basis Alignment and SGD Noise","created":"2025-10-15","cited_by_count":2,"area":null},{"id":"2510.17486","title":"Local properties of neural networks through the lens of layer-wise Hessians","created":"2025-10-20","cited_by_count":2,"area":null},{"id":"2602.05600","title":"On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2605.00650","title":"AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments","created":"2026-05-01","cited_by_count":2,"area":null},{"id":"2201.06964","title":"There is a Singularity in the Loss Landscape","created":"2022-01-12","cited_by_count":1,"area":null},{"id":"2308.06679","title":"Separable Gaussian Neural Networks: Structure, Analysis, and Function Approximations","created":"2023-08-13","cited_by_count":1,"area":null},{"id":"2309.05077","title":"Generalization error bounds for iterative learning algorithms with bounded updates","created":"2023-09-10","cited_by_count":1,"area":null}],"next_cursor":"100"}