{"list":"other","total":51,"rows":[{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2304.07193","title":"DINOv2: Learning Robust Visual Features without Supervision","created":"2023-04-14","cited_by_count":7974},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"1902.00751","title":"Parameter-Efficient Transfer Learning for NLP","created":"2019-02-02","cited_by_count":4236},{"id":"1502.01852","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","created":"2015-02-06","cited_by_count":3611},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1606.04838","title":"Optimization Methods for Large-Scale Machine Learning","created":"2016-06-15","cited_by_count":1150},{"id":"1503.05671","title":"Optimizing Neural Networks with Kronecker-factored Approximate Curvature","created":"2015-03-19","cited_by_count":796},{"id":"1908.03265","title":"On the Variance of the Adaptive Learning Rate and Beyond","created":"2019-08-08","cited_by_count":772},{"id":"2403.14608","title":"Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey","created":"2024-03-21","cited_by_count":656},{"id":"1308.0850","title":"Generating Sequences With Recurrent Neural Networks","created":"2013-08-04","cited_by_count":554},{"id":"2002.08791","title":"Bayesian Deep Learning and a Probabilistic Perspective of Generalization","created":"2020-02-20","cited_by_count":480},{"id":"1412.1193","title":"New insights and perspectives on the natural gradient method","created":"2014-12-03","cited_by_count":443},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"1611.01838","title":"Entropy-SGD: Biasing Gradient Descent Into Wide Valleys","created":"2016-11-06","cited_by_count":359},{"id":"1901.10159","title":"An Investigation into Neural Net Optimization via Hessian Eigenvalue Density","created":"2019-01-29","cited_by_count":292},{"id":"1706.04454","title":"Empirical Analysis of the Hessian of Over-Parametrized Neural Networks","created":"2017-06-14","cited_by_count":290},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1301.3584","title":"Revisiting Natural Gradient for Deep Networks","created":"2013-01-16","cited_by_count":159},{"id":"1611.07476","title":"Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond","created":"2016-11-22","cited_by_count":159},{"id":"2009.11162","title":"Implicit Gradient Regularization","created":"2020-09-23","cited_by_count":146},{"id":"1705.07774","title":"Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients","created":"2017-05-22","cited_by_count":139},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124},{"id":"1506.02617","title":"Path-SGD: Path-Normalized Optimization in Deep Neural Networks","created":"2015-06-08","cited_by_count":120},{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114},{"id":"1710.11029","title":"Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks","created":"2017-10-30","cited_by_count":109},{"id":"2107.04562","title":"The Bayesian Learning Rule","created":"2021-07-09","cited_by_count":99},{"id":"1709.01953","title":"Implicit Regularization in Deep Learning","created":"2017-09-06","cited_by_count":90},{"id":"1706.10239","title":"Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes","created":"2017-06-30","cited_by_count":84},{"id":"1905.10961","title":"Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks","created":"2019-05-27","cited_by_count":76},{"id":"2402.17641","title":"Variational Learning is Effective for Large Deep Networks","created":"2024-02-27","cited_by_count":65},{"id":"2012.14193","title":"Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization","created":"2020-12-28","cited_by_count":62},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53},{"id":"1906.07774","title":"On the interplay between noise and curvature and its effect on optimization and generalization","created":"2019-06-18","cited_by_count":44},{"id":"2012.03801","title":"A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization","created":"2020-12-07","cited_by_count":44},{"id":"1703.00209","title":"Online Natural Gradient as a Kalman Filter","created":"2017-03-01","cited_by_count":25},{"id":"2402.03496","title":"Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective","created":"2024-02-05","cited_by_count":25},{"id":"2305.19535","title":"Low-rank extended Kalman filtering for online learning of neural networks from streaming data","created":"2023-05-31","cited_by_count":23},{"id":"2210.02720","title":"Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias","created":"2022-10-06","cited_by_count":16},{"id":"2212.00781","title":"Second-order optimization with lazy Hessians","created":"2022-12-01","cited_by_count":15},{"id":"1911.08192","title":"Information-Theoretic Local Minima Characterization and Regularization","created":"2019-11-19","cited_by_count":14},{"id":"2405.19681","title":"Bayesian Online Natural Gradient (BONG)","created":"2024-05-30","cited_by_count":14},{"id":"2302.11962","title":"Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods","created":"2023-02-23","cited_by_count":12},{"id":"2405.08971","title":"Computation-Aware Kalman Filtering and Smoothing","created":"2024-05-14","cited_by_count":12},{"id":"1901.00696","title":"The Extended Kalman Filter is a Natural Gradient Descent in Trajectory Space","created":"2019-01-03","cited_by_count":10},{"id":"2311.02495","title":"Uncertainty Quantification in Multivariable Regression for Material Property Prediction with Bayesian Neural Networks","created":"2023-11-04","cited_by_count":5},{"id":"2007.09525","title":"New Proximal Newton-Type Methods for Convex Optimization","created":"2020-07-18","cited_by_count":3},{"id":"2511.01694","title":"Bayesian Natural Gradient Fine-Tuning of CLIP Models via Kalman Filtering","created":"2025-11-03","cited_by_count":1}],"next_cursor":null}