{"list":"other","total":59,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1405.0312","title":"Microsoft COCO: Common Objects in Context","created":"2014-05-01","cited_by_count":17608},{"id":"1409.1556","title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","created":"2014-09-04","cited_by_count":15364},{"id":"1506.01497","title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","created":"2015-06-04","cited_by_count":9639},{"id":"1608.06993","title":"Densely Connected Convolutional Networks","created":"2016-08-25","cited_by_count":6720},{"id":"1606.03498","title":"Improved Techniques for Training GANs","created":"2016-06-10","cited_by_count":2970},{"id":"1611.03530","title":"Understanding deep learning requires rethinking generalization","created":"2016-11-10","cited_by_count":1674},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1606.04838","title":"Optimization Methods for Large-Scale Machine Learning","created":"2016-06-15","cited_by_count":1150},{"id":"1712.09913","title":"Visualizing the Loss Landscape of Neural Nets","created":"2017-12-28","cited_by_count":1130},{"id":"1609.04747","title":"An overview of gradient descent optimization algorithms","created":"2016-09-15","cited_by_count":1033},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"1904.09237","title":"On the Convergence of Adam and Beyond","created":"2019-04-19","cited_by_count":841},{"id":"1908.03265","title":"On the Variance of the Adaptive Learning Rate and Beyond","created":"2019-08-08","cited_by_count":772},{"id":"1701.07875","title":"Wasserstein GAN","created":"2017-01-26","cited_by_count":639},{"id":"1802.04434","title":"signSGD: Compressed Optimisation for Non-Convex Problems","created":"2018-02-13","cited_by_count":571},{"id":"1509.01240","title":"Train faster, generalize better: Stability of stochastic gradient descent","created":"2015-09-03","cited_by_count":552},{"id":"2302.06675","title":"Symbolic Discovery of Optimization Algorithms","created":"2023-02-13","cited_by_count":434},{"id":"1407.0202","title":"SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives","created":"2014-07-01","cited_by_count":402},{"id":"1905.11881","title":"Why gradient clipping accelerates training: A theoretical justification for adaptivity","created":"2019-05-28","cited_by_count":341},{"id":"1704.04289","title":"Stochastic Gradient Descent as Approximate Bayesian Inference","created":"2017-04-13","cited_by_count":300},{"id":"1912.07145","title":"PyHessian: Neural Networks Through the Lens of the Hessian","created":"2019-12-16","cited_by_count":292},{"id":"1406.2572","title":"Identifying and attacking the saddle point problem in high-dimensional non-convex optimization","created":"2014-06-10","cited_by_count":289},{"id":"2010.07468","title":"AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients","created":"2020-10-15","cited_by_count":283},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265},{"id":"1705.08292","title":"The Marginal Value of Adaptive Gradient Methods in Machine Learning","created":"2017-05-23","cited_by_count":250},{"id":"1706.08500","title":"GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium","created":"2017-06-26","cited_by_count":213},{"id":"2006.00719","title":"ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning","created":"2020-06-01","cited_by_count":202},{"id":"1902.09843","title":"Adaptive Gradient Methods with Dynamic Bound of Learning Rate","created":"2019-02-26","cited_by_count":183},{"id":"1808.02941","title":"On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization","created":"2018-08-08","cited_by_count":174},{"id":"2203.08065","title":"Surrogate Gap Minimization Improves Sharpness-Aware Training","created":"2022-03-15","cited_by_count":172},{"id":"1309.2388","title":"Minimizing Finite Sums with the Stochastic Average Gradient","created":"2013-09-10","cited_by_count":164},{"id":"1610.02581","title":"Variance-based regularization with convex objectives","created":"2016-10-08","cited_by_count":155},{"id":"1705.07774","title":"Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients","created":"2017-05-22","cited_by_count":139},{"id":"2002.03305","title":"Momentum Improves Normalized SGD","created":"2020-02-09","cited_by_count":139},{"id":"1712.07628","title":"Improving Generalization Performance by Switching from Adam to SGD","created":"2017-12-20","cited_by_count":125},{"id":"1811.01558","title":"Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations","created":"2018-11-05","cited_by_count":114},{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114},{"id":"1803.01206","title":"On the Power of Over-parametrization in Neural Networks with Quadratic Activation","created":"2018-03-03","cited_by_count":102},{"id":"2203.11834","title":"Improving Generalization in Federated Learning by Seeking Flat Minima","created":"2022-03-22","cited_by_count":98},{"id":"1806.06763","title":"Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks","created":"2018-06-18","cited_by_count":90},{"id":"1802.08241","title":"Hessian-based Analysis of Large Batch Training and Robustness to Adversaries","created":"2018-02-22","cited_by_count":87},{"id":"2303.03108","title":"Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization","created":"2023-03-03","cited_by_count":57},{"id":"2305.18471","title":"Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions","created":"2023-05-29","cited_by_count":57},{"id":"2002.03432","title":"On the distance between two neural networks and the stability of learning","created":"2020-02-09","cited_by_count":46},{"id":"2006.15815","title":"Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum","created":"2020-06-29","cited_by_count":40},{"id":"2002.10376","title":"The Two Regimes of Deep Network Training","created":"2020-02-24","cited_by_count":30},{"id":"2208.09900","title":"Provable Adaptivity of Adam under Non-uniform Smoothness","created":"2022-08-21","cited_by_count":30},{"id":"2206.01246","title":"Stochastic gradient descent introduces an effective landscape-dependent regularization favoring flat solutions","created":"2022-06-02","cited_by_count":22},{"id":"1706.03471","title":"YellowFin and the Art of Momentum Tuning","created":"2017-06-12","cited_by_count":16},{"id":"1901.00696","title":"The Extended Kalman Filter is a Natural Gradient Descent in Trajectory Space","created":"2019-01-03","cited_by_count":10},{"id":"2011.02150","title":"EAdam Optimizer: How $\\epsilon$ Impact Adam","created":"2020-11-04","cited_by_count":10},{"id":"2208.07951","title":"On the generalization of learning algorithms that do not converge","created":"2022-08-16","cited_by_count":9},{"id":"1810.12273","title":"Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization","created":"2018-10-29","cited_by_count":8},{"id":"2107.03331","title":"KOALA: A Kalman Optimization Algorithm with Loss Adaptivity","created":"2021-07-07","cited_by_count":6},{"id":"2209.09162","title":"On the Theoretical Properties of Noise Correlation in Stochastic Optimization","created":"2022-09-19","cited_by_count":5},{"id":"2211.03970","title":"On the Algorithmic Stability and Generalization of Adaptive Optimization Methods","created":"2022-11-08","cited_by_count":5}],"next_cursor":null}