{"list":"other","total":54,"rows":[{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1409.1556","title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","created":"2014-09-04","cited_by_count":15364},{"id":"1502.03167","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","created":"2015-02-11","cited_by_count":7005},{"id":"1803.03635","title":"The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks","created":"2018-03-09","cited_by_count":2262},{"id":"1703.04730","title":"Understanding Black-box Predictions via Influence Functions","created":"2017-03-14","cited_by_count":1741},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"2301.05217","title":"Progress measures for grokking via mechanistic interpretability","created":"2023-01-12","cited_by_count":839},{"id":"1312.6120","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","created":"2013-12-20","cited_by_count":763},{"id":"1912.02292","title":"Deep Double Descent: Where Bigger Models and More Data Hurt","created":"2019-12-04","cited_by_count":672},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"2302.06675","title":"Symbolic Discovery of Optimization Algorithms","created":"2023-02-13","cited_by_count":434},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"2009.10795","title":"Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics","created":"2020-09-22","cited_by_count":425},{"id":"1805.11604","title":"How Does Batch Normalization Help Optimization?","created":"2018-05-29","cited_by_count":357},{"id":"2006.07710","title":"The Pitfalls of Simplicity Bias in Neural Networks","created":"2020-06-13","cited_by_count":351},{"id":"1901.10159","title":"An Investigation into Neural Net Optimization via Hessian Eigenvalue Density","created":"2019-01-29","cited_by_count":292},{"id":"1706.04454","title":"Empirical Analysis of the Hessian of Over-Parametrized Neural Networks","created":"2017-06-14","cited_by_count":290},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"1905.11604","title":"SGD on Neural Networks Learns Functions of Increasing Complexity","created":"2019-05-28","cited_by_count":204},{"id":"2207.08799","title":"Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit","created":"2022-07-18","cited_by_count":187},{"id":"1805.08522","title":"Deep learning generalizes because the parameter-function map is biased towards simple functions","created":"2018-05-22","cited_by_count":183},{"id":"1806.00900","title":"Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced","created":"2018-06-04","cited_by_count":177},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"1611.07476","title":"Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond","created":"2016-11-22","cited_by_count":159},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114},{"id":"2207.14484","title":"Adaptive Gradient Methods at the Edge of Stability","created":"2022-07-29","cited_by_count":101},{"id":"2008.11865","title":"Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra","created":"2020-08-27","cited_by_count":94},{"id":"2206.04817","title":"The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon","created":"2022-06-10","cited_by_count":94},{"id":"1807.05031","title":"On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length","created":"2018-07-13","cited_by_count":88},{"id":"2303.11873","title":"A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks","created":"2023-03-21","cited_by_count":83},{"id":"1802.08770","title":"A Walk with SGD","created":"2018-02-24","cited_by_count":78},{"id":"2202.06856","title":"Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization","created":"2022-02-14","cited_by_count":69},{"id":"1901.08244","title":"Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians","created":"2019-01-24","cited_by_count":68},{"id":"1906.05392","title":"Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian","created":"2019-06-12","cited_by_count":67},{"id":"2207.02628","title":"The alignment property of SGD noise and how it helps select flat minima: A stability analysis","created":"2022-07-06","cited_by_count":64},{"id":"2105.13462","title":"On Linear Stability of SGD and Input-Smoothness of Neural Networks","created":"2021-05-27","cited_by_count":63},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63},{"id":"2012.14193","title":"Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization","created":"2020-12-28","cited_by_count":62},{"id":"2210.03294","title":"Understanding Edge-of-Stability Training Dynamics with a Minimalist Example","created":"2022-10-07","cited_by_count":56},{"id":"2303.08114","title":"Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs","created":"2023-03-14","cited_by_count":53},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53},{"id":"1910.05929","title":"Emergent properties of the local geometry of neural loss landscapes","created":"2019-10-14","cited_by_count":46},{"id":"2204.11326","title":"Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes","created":"2022-04-24","cited_by_count":43},{"id":"1907.10732","title":"Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization","created":"2019-07-24","cited_by_count":37},{"id":"2306.13253","title":"Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok","created":"2023-06-23","cited_by_count":35},{"id":"2201.12250","title":"Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization","created":"2022-01-28","cited_by_count":30},{"id":"2301.12309","title":"On the Lipschitz Constant of Deep Networks and Double Descent","created":"2023-01-28","cited_by_count":11},{"id":"2305.14683","title":"On progressive sharpening, flat minima and generalisation","created":"2023-05-24","cited_by_count":7}],"next_cursor":null}