{"list":"other","total":53,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1912.01703","title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","created":"2019-12-03","cited_by_count":20305},{"id":"2103.14030","title":"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows","created":"2021-03-25","cited_by_count":12139},{"id":"1409.0575","title":"ImageNet Large Scale Visual Recognition Challenge","created":"2014-09-01","cited_by_count":8498},{"id":"1502.03167","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","created":"2015-02-11","cited_by_count":7005},{"id":"1608.06993","title":"Densely Connected Convolutional Networks","created":"2016-08-25","cited_by_count":6720},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"1608.03983","title":"SGDR: Stochastic Gradient Descent with Warm Restarts","created":"2016-08-13","cited_by_count":5045},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2106.01946","title":"Convex optimization","created":"2021-06-03","cited_by_count":2515},{"id":"1905.02244","title":"Searching for MobileNetV3","created":"2019-05-06","cited_by_count":1994},{"id":"1708.04552","title":"Improved Regularization of Convolutional Neural Networks with Cutout","created":"2017-08-15","cited_by_count":1628},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1609.04747","title":"An overview of gradient descent optimization algorithms","created":"2016-09-15","cited_by_count":1033},{"id":"1805.07836","title":"Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels","created":"2018-05-20","cited_by_count":997},{"id":"1804.04235","title":"Adafactor: Adaptive Learning Rates with Sublinear Memory Cost","created":"2018-04-11","cited_by_count":905},{"id":"1904.09237","title":"On the Convergence of Adam and Beyond","created":"2019-04-19","cited_by_count":841},{"id":"1503.05671","title":"Optimizing Neural Networks with Kronecker-factored Approximate Curvature","created":"2015-03-19","cited_by_count":796},{"id":"1908.03265","title":"On the Variance of the Adaptive Learning Rate and Beyond","created":"2019-08-08","cited_by_count":772},{"id":"1904.00962","title":"Large Batch Optimization for Deep Learning: Training BERT in 76 minutes","created":"2019-04-01","cited_by_count":674},{"id":"1708.03888","title":"Large Batch Training of Convolutional Networks","created":"2017-08-13","cited_by_count":535},{"id":"1412.1193","title":"New insights and perspectives on the natural gradient method","created":"2014-12-03","cited_by_count":443},{"id":"2102.08604","title":"SWAD: Domain Generalization by Seeking Flat Minima","created":"2021-02-17","cited_by_count":434},{"id":"2302.06675","title":"Symbolic Discovery of Optimization Algorithms","created":"2023-02-13","cited_by_count":434},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"2110.00476","title":"ResNet strikes back: An improved training procedure in timm","created":"2021-10-01","cited_by_count":368},{"id":"2010.07468","title":"AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients","created":"2020-10-15","cited_by_count":283},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265},{"id":"2104.05704","title":"Escaping the Big Data Paradigm with Compact Transformers","created":"2021-04-12","cited_by_count":264},{"id":"1705.08292","title":"The Marginal Value of Adaptive Gradient Methods in Machine Learning","created":"2017-05-23","cited_by_count":250},{"id":"1905.12558","title":"Limitations of the Empirical Fisher Approximation for Natural Gradient Descent","created":"2019-05-29","cited_by_count":217},{"id":"2006.00719","title":"ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning","created":"2020-06-01","cited_by_count":202},{"id":"2203.11926","title":"Focal Modulation Networks","created":"2022-03-22","cited_by_count":186},{"id":"1902.09843","title":"Adaptive Gradient Methods with Dynamic Bound of Learning Rate","created":"2019-02-26","cited_by_count":183},{"id":"1808.02941","title":"On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization","created":"2018-08-08","cited_by_count":174},{"id":"1602.01407","title":"A Kronecker-factored approximate Fisher matrix for convolution layers","created":"2016-02-03","cited_by_count":141},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124},{"id":"2205.10287","title":"On the SDEs and Scaling Rules for Adaptive Gradient Algorithms","created":"2022-05-20","cited_by_count":104},{"id":"2009.12836","title":"Normalization Techniques in Training DNNs: Methodology, Analysis and Application","created":"2020-09-27","cited_by_count":90},{"id":"1811.09030","title":"Data Augmentation using Random Image Cropping and Patching for Deep CNNs","created":"2018-11-22","cited_by_count":88},{"id":"2311.00636","title":"Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures","created":"2023-11-01","cited_by_count":49},{"id":"2201.12250","title":"Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization","created":"2022-01-28","cited_by_count":30},{"id":"2305.14858","title":"Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers","created":"2023-05-24","cited_by_count":30},{"id":"2402.03496","title":"Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective","created":"2024-02-05","cited_by_count":25},{"id":"2211.02254","title":"How Does Adaptive Optimization Impact Local Neural Network Geometry?","created":"2022-11-04","cited_by_count":24},{"id":"2305.04684","title":"ASDL: A Unified Interface for Gradient Preconditioning in PyTorch","created":"2023-05-08","cited_by_count":21},{"id":"2209.15168","title":"Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification","created":"2022-09-30","cited_by_count":10},{"id":"1903.06237","title":"Inefficiency of K-FAC for Large Batch Size Training","created":"2019-03-14","cited_by_count":4},{"id":"2403.02867","title":"Scalable Continuous-time Diffusion Framework for Network Inference and Influence Estimation","created":"2024-03-05","cited_by_count":4},{"id":"2211.15596","title":"A survey of deep learning optimizers -- first and second order methods","created":"2022-11-28","cited_by_count":3},{"id":"2109.12504","title":"AdaInject: Injection Based Adaptive Gradient Descent Optimizers for Convolutional Neural Networks","created":"2021-09-26","cited_by_count":2},{"id":"2209.14937","title":"NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizer","created":"2022-09-29","cited_by_count":2}],"next_cursor":null}