{"total":263,"rows":[{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427,"area":null},{"id":"2305.17333","title":"Fine-Tuning Language Models with Just Forward Passes","created":"2023-05-27","cited_by_count":345,"area":null},{"id":"2202.06709","title":"How Do Vision Transformers Work?","created":"2022-02-14","cited_by_count":307,"area":null},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265,"area":null},{"id":"2109.02934","title":"Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization","created":"2021-09-07","cited_by_count":186,"area":null},{"id":"2303.01486","title":"Understanding plasticity in neural networks","created":"2023-03-02","cited_by_count":174,"area":null},{"id":"2303.06296","title":"Stabilizing Transformer Training by Preventing Attention Entropy Collapse","created":"2023-03-11","cited_by_count":161,"area":null},{"id":"2106.09647","title":"Deep Learning Through the Lens of Example Difficulty","created":"2021-06-17","cited_by_count":141,"area":null},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137,"area":null},{"id":"2112.03052","title":"Scaling Up Influence Functions","created":"2021-12-06","cited_by_count":132,"area":null},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2402.01868","title":"Challenges in Training PINNs: A Loss Landscape Perspective","created":"2024-02-02","cited_by_count":113,"area":null},{"id":"2111.10603","title":"Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning","created":"2021-11-20","cited_by_count":101,"area":null},{"id":"2206.04041","title":"Neural Collapse: A Review on Modelling Principles and Generalization","created":"2022-06-08","cited_by_count":95,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2206.08213","title":"A Closer Look at Smoothness in Domain Adversarial Training","created":"2022-06-16","cited_by_count":85,"area":null},{"id":"2202.00661","title":"When Do Flat Minima Optimizers Work?","created":"2022-02-01","cited_by_count":78,"area":null},{"id":"2210.05177","title":"Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach","created":"2022-10-11","cited_by_count":70,"area":null},{"id":"2402.15173","title":"Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer","created":"2024-02-23","cited_by_count":53,"area":null},{"id":"2403.12350","title":"Friendly Sharpness-Aware Minimization","created":"2024-03-19","cited_by_count":53,"area":null},{"id":"2106.16225","title":"Analytic Insights into Structure and Rank of Neural Network Hessian Maps","created":"2021-06-30","cited_by_count":52,"area":null},{"id":"2207.00160","title":"When Does Differentially Private Learning Not Suffer in High Dimensions?","created":"2022-07-01","cited_by_count":48,"area":null},{"id":"2106.00042","title":"A study on the plasticity of neural networks","created":"2021-05-31","cited_by_count":44,"area":null},{"id":"2110.04369","title":"A Loss Curvature Perspective on Training Instability in Deep Learning","created":"2021-10-08","cited_by_count":40,"area":null},{"id":"2210.04860","title":"Second-order regression models exhibit progressive sharpening to the edge of stability","created":"2022-10-10","cited_by_count":40,"area":null},{"id":"2201.08025","title":"Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape","created":"2022-01-20","cited_by_count":39,"area":null},{"id":"2309.15639","title":"Enhancing Sharpness-Aware Optimization Through Variance Suppression","created":"2023-09-27","cited_by_count":37,"area":null},{"id":"2405.16002","title":"Does SGD really happen in tiny subspaces?","created":"2024-05-25","cited_by_count":36,"area":null},{"id":"2102.04396","title":"SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality","created":"2021-02-08","cited_by_count":35,"area":null},{"id":"2206.02659","title":"Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees","created":"2022-06-06","cited_by_count":34,"area":null},{"id":"2103.01519","title":"Hessian Eigenspectra of More Realistic Nonlinear Models","created":"2021-03-02","cited_by_count":33,"area":null},{"id":"2405.04230","title":"Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?","created":"2024-05-07","cited_by_count":31,"area":null},{"id":"2305.16971","title":"Theoretical and Practical Perspectives on what Influence Functions Do","created":"2023-05-26","cited_by_count":29,"area":null},{"id":"2107.05802","title":"How many degrees of freedom do we need to train deep networks: a loss landscape perspective","created":"2021-07-13","cited_by_count":28,"area":null},{"id":"2212.13827","title":"Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data","created":"2022-12-28","cited_by_count":28,"area":null},{"id":"2307.05831","title":"Memorization Through the Lens of Curvature of Loss Function Around Samples","created":"2023-07-11","cited_by_count":27,"area":null},{"id":"2310.05801","title":"An operator preconditioning perspective on training in physics-informed machine learning","created":"2023-10-09","cited_by_count":27,"area":null},{"id":"2201.00519","title":"Stochastic Weight Averaging Revisited","created":"2022-01-03","cited_by_count":26,"area":null},{"id":"2310.09639","title":"DPZero: Private Fine-Tuning of Language Models without Backpropagation","created":"2023-10-14","cited_by_count":26,"area":null},{"id":"2312.00246","title":"Directions of Curvature as an Explanation for Loss of Plasticity","created":"2023-11-30","cited_by_count":26,"area":null},{"id":"2309.16932","title":"Symmetry Induces Structure and Constraint of Learning","created":"2023-09-29","cited_by_count":25,"area":null},{"id":"2312.15184","title":"ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and Uncertainty in Zeroth-order Optimization","created":"2023-12-23","cited_by_count":25,"area":null},{"id":"2302.08692","title":"SAM operates far from home: eigenvalue regularization as a dynamical phenomenon","created":"2023-02-17","cited_by_count":24,"area":null},{"id":"2308.00261","title":"Improving Pixel-based MIM by Reducing Wasted Modeling Capability","created":"2023-08-01","cited_by_count":24,"area":null},{"id":"2308.02533","title":"Improving Generalization of Adversarial Training via Robust Critical Fine-Tuning","created":"2023-08-01","cited_by_count":23,"area":null},{"id":"2402.17457","title":"Super Consistency of Neural Network Landscapes and Learning Rate Transfer","created":"2024-02-27","cited_by_count":23,"area":null},{"id":"2301.02312","title":"Training trajectories, mini-batch losses and the curious role of the learning rate","created":"2023-01-05","cited_by_count":21,"area":null},{"id":"2106.15739","title":"On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay","created":"2021-06-29","cited_by_count":20,"area":null},{"id":"2303.12922","title":"Revisiting the Fragility of Influence Functions","created":"2023-03-22","cited_by_count":20,"area":null},{"id":"2107.09133","title":"The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion","created":"2021-07-19","cited_by_count":19,"area":null},{"id":"2402.05926","title":"On the Convergence of Zeroth-Order Federated Tuning for Large Language Models","created":"2024-02-08","cited_by_count":19,"area":null},{"id":"2110.11749","title":"Feature Learning and Signal Propagation in Deep Neural Networks","created":"2021-10-22","cited_by_count":18,"area":null},{"id":"2404.19261","title":"High dimensional analysis reveals conservative sharpening and a stochastic edge of stability","created":"2024-04-30","cited_by_count":18,"area":null},{"id":"2206.11925","title":"Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets","created":"2022-06-23","cited_by_count":17,"area":null},{"id":"2302.03764","title":"Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions","created":"2023-02-07","cited_by_count":17,"area":null},{"id":"2309.12488","title":"Sharpness-Aware Minimization and the Edge of Stability","created":"2023-09-21","cited_by_count":17,"area":null},{"id":"2101.04968","title":"Learning with Gradient Descent and Weakly Convex Losses","created":"2021-01-13","cited_by_count":16,"area":null},{"id":"2106.02624","title":"ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure","created":"2021-06-04","cited_by_count":16,"area":null},{"id":"2106.03696","title":"Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models","created":"2021-06-07","cited_by_count":16,"area":null},{"id":"2107.10370","title":"Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II","created":"2021-07-21","cited_by_count":16,"area":null},{"id":"2201.13011","title":"On the Power-Law Hessian Spectrums in Deep Learning","created":"2022-01-31","cited_by_count":16,"area":null},{"id":"2202.00280","title":"Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?","created":"2022-02-01","cited_by_count":16,"area":null},{"id":"2603.20527","title":"RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization","created":"2026-03-20","cited_by_count":15,"area":null},{"id":"2105.12639","title":"Blurs Behave Like Ensembles: Spatial Smoothings to Improve Accuracy, Uncertainty, and Robustness","created":"2021-05-26","cited_by_count":14,"area":null},{"id":"2210.01407","title":"Homotopy-based training of NeuralODEs for accurate dynamics discovery","created":"2022-10-04","cited_by_count":14,"area":null},{"id":"2505.01660","title":"Focal-SAM: Focal Sharpness-Aware Minimization for Long-Tailed Classification","created":"2025-05-03","cited_by_count":14,"area":null},{"id":"2509.26544","title":"Bayesian Influence Functions for Hessian-Free Data Attribution","created":"2025-09-30","cited_by_count":14,"area":null},{"id":"2607.21716","title":"A Defense of the Quadratic Model","created":"2026-07-23","cited_by_count":14,"area":null},{"id":"2106.12718","title":"Sparse Flows: Pruning Continuous-depth Models","created":"2021-06-24","cited_by_count":13,"area":null},{"id":"2210.06041","title":"Reinforcement Learning with Automated Auxiliary Loss Search","created":"2022-10-12","cited_by_count":13,"area":null},{"id":"2212.02083","title":"On the Overlooked Structure of Stochastic Gradients","created":"2022-12-05","cited_by_count":13,"area":null},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13,"area":null},{"id":"2404.00357","title":"Revisiting Random Weight Perturbation for Efficiently Improving Generalization","created":"2024-03-30","cited_by_count":13,"area":null},{"id":"2405.15861","title":"Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization","created":"2024-05-24","cited_by_count":13,"area":null},{"id":"2406.04284","title":"What is Dataset Distillation Learning?","created":"2024-06-06","cited_by_count":13,"area":null},{"id":"2505.02809","title":"Towards Quantifying the Hessian Structure of Neural Networks","created":"2025-05-05","cited_by_count":13,"area":null},{"id":"2602.11948","title":"Insights on Muon from Simple Quadratics","created":"2026-02-12","cited_by_count":13,"area":null},{"id":"2106.03743","title":"Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence","created":"2021-06-07","cited_by_count":12,"area":null},{"id":"2310.16391","title":"Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization","created":"2023-10-25","cited_by_count":12,"area":null},{"id":"2104.00365","title":"Federated Few-Shot Learning with Adversarial Learning","created":"2021-04-01","cited_by_count":11,"area":null},{"id":"2302.01952","title":"On a continuous time model of gradient descent dynamics and instability in deep learning","created":"2023-02-03","cited_by_count":11,"area":null},{"id":"2404.15593","title":"A Survey of Deep Long-Tail Classification Advancements","created":"2024-04-24","cited_by_count":11,"area":null},{"id":"2410.04779","title":"Fast Training of Sinusoidal Neural Fields via Scaling Initialization","created":"2024-10-07","cited_by_count":11,"area":null},{"id":"2305.09088","title":"The Hessian perspective into the Nature of Convolutional Neural Networks","created":"2023-05-16","cited_by_count":10,"area":null},{"id":"2402.03994","title":"Efficient Sketches for Training Data Attribution and Studying the Loss Landscape","created":"2024-02-06","cited_by_count":10,"area":null},{"id":"2507.14740","title":"Better Training Data Attribution via Better Inverse Hessian-Vector Products","created":"2025-07-19","cited_by_count":10,"area":null},{"id":"2602.00170","title":"The Blessing of Dimensionality in LLM Fine-tuning: A Variance-Curvature Perspective","created":"2026-01-30","cited_by_count":10,"area":null},{"id":"2107.09194","title":"Can we globally optimize cross-validation loss? Quasiconvexity in ridge regression","created":"2021-07-19","cited_by_count":9,"area":null},{"id":"2209.02525","title":"Generalisation under gradient descent via deterministic PAC-Bayes","created":"2022-09-06","cited_by_count":9,"area":null},{"id":"2303.15421","title":"ACAT: Adversarial Counterfactual Attention for Classification and Detection in Medical Imaging","created":"2023-03-27","cited_by_count":9,"area":null},{"id":"2406.01255","title":"On the Nonlinearity of Layer Normalization","created":"2024-06-03","cited_by_count":9,"area":null},{"id":"2410.10986","title":"What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis","created":"2024-10-14","cited_by_count":9,"area":null},{"id":"2302.09726","title":"Nystrom Method for Accurate and Scalable Implicit Differentiation","created":"2023-02-20","cited_by_count":8,"area":null},{"id":"2305.01134","title":"PGrad: Learning Principal Gradients For Domain Generalization","created":"2023-05-02","cited_by_count":8,"area":null},{"id":"2407.06183","title":"Stepping on the Edge: Curvature Aware Learning Rate Tuners","created":"2024-07-08","cited_by_count":8,"area":null},{"id":"2602.18584","title":"GIST: Targeted Data Selection for Instruction Tuning via Coupled Optimization Geometry","created":"2026-02-20","cited_by_count":8,"area":null},{"id":"2602.22681","title":"Accelerating LLM Pre-Training through Flat-Direction Dynamics Enhancement","created":"2026-02-26","cited_by_count":8,"area":null},{"id":"2102.03793","title":"Tilting the playing field: Dynamical loss functions for machine learning","created":"2021-02-07","cited_by_count":7,"area":null},{"id":"2102.06604","title":"Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks","created":"2021-02-12","cited_by_count":7,"area":null},{"id":"2102.06740","title":"Appearance of Random Matrix Theory in Deep Learning","created":"2021-02-12","cited_by_count":7,"area":null}],"next_cursor":"100"}