{"list":"other","total":62,"rows":[{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2403.05530","title":"Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context","created":"2024-03-08","cited_by_count":3224},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1503.05671","title":"Optimizing Neural Networks with Kronecker-factored Approximate Curvature","created":"2015-03-19","cited_by_count":796},{"id":"1912.02178","title":"Fantastic Generalization Measures and Where to Find Them","created":"2019-12-04","cited_by_count":513},{"id":"2502.16982","title":"Muon is Scalable for LLM Training","created":"2025-02-24","cited_by_count":428},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"1611.01838","title":"Entropy-SGD: Biasing Gradient Descent Into Wide Valleys","created":"2016-11-06","cited_by_count":359},{"id":"1803.00885","title":"Essentially No Barriers in Neural Network Energy Landscape","created":"2018-03-02","cited_by_count":330},{"id":"2202.06709","title":"How Do Vision Transformers Work?","created":"2022-02-14","cited_by_count":307},{"id":"1901.10159","title":"An Investigation into Neural Net Optimization via Hessian Eigenvalue Density","created":"2019-01-29","cited_by_count":292},{"id":"1912.07145","title":"PyHessian: Neural Networks Through the Lens of the Hessian","created":"2019-12-16","cited_by_count":292},{"id":"1706.04454","title":"Empirical Analysis of the Hessian of Over-Parametrized Neural Networks","created":"2017-06-14","cited_by_count":290},{"id":"1406.2572","title":"Identifying and attacking the saddle point problem in high-dimensional non-convex optimization","created":"2014-06-10","cited_by_count":289},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265},{"id":"1812.04754","title":"Gradient Descent Happens in a Tiny Subspace","created":"2018-12-12","cited_by_count":217},{"id":"2409.11321","title":"SOAP: Improving and Stabilizing Shampoo using Adam","created":"2024-09-17","cited_by_count":192},{"id":"1611.07476","title":"Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond","created":"2016-11-22","cited_by_count":159},{"id":"1903.11680","title":"Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks","created":"2019-03-27","cited_by_count":159},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"1806.03884","title":"Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis","created":"2018-06-11","cited_by_count":146},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124},{"id":"1907.04164","title":"Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model","created":"2019-07-09","cited_by_count":117},{"id":"2207.14484","title":"Adaptive Gradient Methods at the Edge of Stability","created":"2022-07-29","cited_by_count":101},{"id":"2008.11865","title":"Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra","created":"2020-08-27","cited_by_count":94},{"id":"1902.00744","title":"Asymmetric Valleys: Beyond Sharp and Flat Local Minima","created":"2019-02-02","cited_by_count":90},{"id":"1807.05031","title":"On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length","created":"2018-07-13","cited_by_count":88},{"id":"1802.08241","title":"Hessian-based Analysis of Large Batch Training and Robustness to Adversaries","created":"2018-02-22","cited_by_count":87},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86},{"id":"1706.10239","title":"Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes","created":"2017-06-30","cited_by_count":84},{"id":"2402.19449","title":"Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models","created":"2024-02-29","cited_by_count":80},{"id":"1912.08957","title":"Optimization for deep learning: theory and algorithms","created":"2019-12-19","cited_by_count":70},{"id":"1901.08244","title":"Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians","created":"2019-01-24","cited_by_count":68},{"id":"2006.08877","title":"Practical Quasi-Newton Methods for Training Deep Neural Networks","created":"2020-06-16","cited_by_count":68},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63},{"id":"2503.20762","title":"ASGO: Adaptive Structured Gradient Optimization","created":"2025-03-26","cited_by_count":57},{"id":"2106.16225","title":"Analytic Insights into Structure and Rank of Neural Network Hessian Maps","created":"2021-06-30","cited_by_count":52},{"id":"2006.09092","title":"Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training","created":"2020-06-16","cited_by_count":50},{"id":"1812.05994","title":"Products of Many Large Random Matrices and Gradients in Deep Neural Networks","created":"2018-12-14","cited_by_count":49},{"id":"1910.05929","title":"Emergent properties of the local geometry of neural loss landscapes","created":"2019-10-14","cited_by_count":46},{"id":"2010.04261","title":"Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks","created":"2020-10-08","cited_by_count":45},{"id":"1507.00210","title":"Natural Neural Networks","created":"2015-07-01","cited_by_count":44},{"id":"2012.03801","title":"A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization","created":"2020-12-07","cited_by_count":44},{"id":"1907.10732","title":"Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization","created":"2019-07-24","cited_by_count":37},{"id":"2103.01519","title":"Hessian Eigenspectra of More Realistic Nonlinear Models","created":"2021-03-02","cited_by_count":33},{"id":"2502.19002","title":"The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training","created":"2025-02-26","cited_by_count":30},{"id":"2411.17525","title":"Pushing the Limits of Large Language Model Quantization via the Linearity Theorem","created":"2024-11-26","cited_by_count":21},{"id":"1902.02366","title":"Negative eigenvalues of the Hessian in deep neural networks","created":"2019-02-06","cited_by_count":20},{"id":"2402.07114","title":"Towards Quantifying the Preconditioning Effect of Adam","created":"2024-02-11","cited_by_count":19},{"id":"1910.00195","title":"How noise affects the Hessian spectrum in overparameterized neural networks","created":"2019-10-01","cited_by_count":17},{"id":"2001.06188","title":"On Random Matrices Arising in Deep Neural Networks. Gaussian Case","created":"2020-01-17","cited_by_count":14},{"id":"1902.01813","title":"Modular Block-diagonal Curvature Approximations for Feedforward Architectures","created":"2019-02-05","cited_by_count":13},{"id":"2410.19964","title":"Understanding Adam Requires Better Rotation Dependent Assumptions","created":"2024-10-25","cited_by_count":11},{"id":"1712.07296","title":"Block-diagonal Hessian-free Optimization for Training Neural Networks","created":"2017-12-20","cited_by_count":10},{"id":"2011.11439","title":"On Random Matrices Arising in Deep Neural Networks: General I.I.D. Case","created":"2020-11-20","cited_by_count":10},{"id":"2410.10986","title":"What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis","created":"2024-10-14","cited_by_count":9},{"id":"2209.00809","title":"Optimal Diagonal Preconditioning","created":"2022-09-02","cited_by_count":7},{"id":"2103.13466","title":"Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case","created":"2021-03-24","cited_by_count":5},{"id":"2201.04543","title":"Eigenvalue Distribution of Large Random Matrices Arising in Deep Neural Networks: Orthogonal Case","created":"2022-01-12","cited_by_count":4}],"next_cursor":null}