{"list":"other","total":87,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2103.14030","title":"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows","created":"2021-03-25","cited_by_count":12139},{"id":"1409.0575","title":"ImageNet Large Scale Visual Recognition Challenge","created":"2014-09-01","cited_by_count":8498},{"id":"1602.05629","title":"Communication-Efficient Learning of Deep Networks from Decentralized Data","created":"2016-02-17","cited_by_count":7669},{"id":"1608.06993","title":"Densely Connected Convolutional Networks","created":"2016-08-25","cited_by_count":6720},{"id":"2201.03545","title":"A ConvNet for the 2020s","created":"2022-01-10","cited_by_count":3579},{"id":"1912.04977","title":"Advances and Open Problems in Federated Learning","created":"2019-12-10","cited_by_count":3046},{"id":"1812.06127","title":"Federated Optimization in Heterogeneous Networks","created":"2018-12-14","cited_by_count":2987},{"id":"1611.03530","title":"Understanding deep learning requires rethinking generalization","created":"2016-11-10","cited_by_count":1674},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"1610.05492","title":"Federated Learning: Strategies for Improving Communication Efficiency","created":"2016-10-18","cited_by_count":1475},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"2203.05482","title":"Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time","created":"2022-03-10","cited_by_count":1396},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1907.02189","title":"On the Convergence of FedAvg on Non-IID Data","created":"2019-07-04","cited_by_count":1229},{"id":"1803.05407","title":"Averaging Weights Leads to Wider Optima and Better Generalization","created":"2018-03-14","cited_by_count":1209},{"id":"2003.00295","title":"Adaptive Federated Optimization","created":"2020-02-29","cited_by_count":1046},{"id":"1904.00962","title":"Large Batch Optimization for Deep Learning: Training BERT in 76 minutes","created":"2019-04-01","cited_by_count":674},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"1912.05671","title":"Linear Mode Connectivity and the Lottery Ticket Hypothesis","created":"2019-12-11","cited_by_count":637},{"id":"1805.09767","title":"Local SGD Converges Fast and Communicates Little","created":"2018-05-24","cited_by_count":605},{"id":"1706.08947","title":"Exploring Generalization in Deep Learning","created":"2017-06-27","cited_by_count":600},{"id":"1802.10026","title":"Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs","created":"2018-02-27","cited_by_count":588},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1803.00885","title":"Essentially No Barriers in Neural Network Energy Landscape","created":"2018-03-02","cited_by_count":330},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"1807.06629","title":"Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning","created":"2018-07-17","cited_by_count":278},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"1909.04746","title":"Tighter Theory for Local SGD on Identical and Heterogeneous Data","created":"2019-09-10","cited_by_count":266},{"id":"1404.5997","title":"One weird trick for parallelizing convolutional neural networks","created":"2014-04-23","cited_by_count":264},{"id":"2106.01548","title":"When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations","created":"2021-06-03","cited_by_count":260},{"id":"1808.07217","title":"Don't Use Large Mini-Batches, Use Local SGD","created":"2018-08-22","cited_by_count":202},{"id":"1811.03600","title":"Measuring the Effects of Data Parallelism on Neural Network Training","created":"2018-11-08","cited_by_count":193},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1906.02367","title":"Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations","created":"2019-06-06","cited_by_count":181},{"id":"2006.06657","title":"Directional convergence and alignment in deep learning","created":"2020-06-11","cited_by_count":178},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155},{"id":"2002.07839","title":"Is Local SGD Better than Minibatch SGD?","created":"2020-02-18","cited_by_count":151},{"id":"2012.09839","title":"Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning","created":"2020-12-17","cited_by_count":150},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"1904.09080","title":"Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process","created":"2019-04-19","cited_by_count":123},{"id":"1910.00643","title":"SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum","created":"2019-10-01","cited_by_count":121},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121},{"id":"2005.06398","title":"Implicit Regularization in Deep Learning May Not Be Explainable by Norms","created":"2020-05-13","cited_by_count":119},{"id":"1802.06175","title":"An Alternative View: When Does SGD Escape Local Minima?","created":"2018-02-17","cited_by_count":115},{"id":"1811.01558","title":"Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations","created":"2018-11-05","cited_by_count":114},{"id":"1910.13598","title":"Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization","created":"2019-10-30","cited_by_count":107},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107},{"id":"2205.10287","title":"On the SDEs and Scaling Rules for Adaptive Gradient Algorithms","created":"2022-05-20","cited_by_count":104},{"id":"2205.01580","title":"Better plain ViT baselines for ImageNet-1k","created":"2022-05-03","cited_by_count":98},{"id":"1708.01012","title":"On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization","created":"2017-08-03","cited_by_count":92},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86},{"id":"2210.14199","title":"Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models","created":"2022-10-25","cited_by_count":84},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83},{"id":"2006.15081","title":"On the Generalization Benefit of Noise in Stochastic Gradient Descent","created":"2020-06-26","cited_by_count":82},{"id":"2104.07705","title":"How to Train BERT with an Academic Budget","created":"2021-04-15","cited_by_count":80},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78},{"id":"1810.08313","title":"Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD","created":"2018-10-19","cited_by_count":77},{"id":"2102.02888","title":"1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed","created":"2021-02-04","cited_by_count":75},{"id":"2102.12470","title":"On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)","created":"2021-02-24","cited_by_count":73},{"id":"2105.13462","title":"On Linear Stability of SGD and Input-Smoothness of Neural Networks","created":"2021-05-27","cited_by_count":63},{"id":"2001.02312","title":"Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well","created":"2020-01-07","cited_by_count":57},{"id":"1904.08913","title":"Deep Rigid Instance Scene Flow","created":"2019-04-18","cited_by_count":54},{"id":"1905.07325","title":"Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models","created":"2019-05-17","cited_by_count":53},{"id":"2006.09092","title":"Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training","created":"2020-06-16","cited_by_count":50},{"id":"2306.12517","title":"FFCV: Accelerating Training by Removing Data Bottlenecks","created":"2023-06-21","cited_by_count":49},{"id":"2204.11326","title":"Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes","created":"2022-04-24","cited_by_count":43},{"id":"2301.11500","title":"Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing","created":"2023-01-27","cited_by_count":42},{"id":"1905.04753","title":"Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints","created":"2019-05-12","cited_by_count":37},{"id":"1910.12308","title":"Asynchronous Decentralized SGD with Quantized and Local Updates","created":"2019-10-27","cited_by_count":36},{"id":"2201.11729","title":"Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks","created":"2022-01-27","cited_by_count":27},{"id":"2303.01215","title":"Why (and When) does Local SGD Generalize Better than SGD?","created":"2023-03-02","cited_by_count":26},{"id":"2104.06069","title":"1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed","created":"2021-04-13","cited_by_count":24},{"id":"2106.06573","title":"Understanding Deflation Process in Over-parametrized Tensor Decomposition","created":"2021-06-11","cited_by_count":20},{"id":"2110.08133","title":"Trade-offs of Local SGD at Scale: An Empirical Study","created":"2021-10-15","cited_by_count":20},{"id":"2210.11948","title":"lo-fi: distributed fine-tuning without communication","created":"2022-10-19","cited_by_count":19},{"id":"2307.15196","title":"The Marginal Value of Momentum for Small Learning Rate SGD","created":"2023-07-27","cited_by_count":18},{"id":"2306.02913","title":"Decentralized SGD and Average-direction SAM are Asymptotically Equivalent","created":"2023-06-05","cited_by_count":15},{"id":"1507.01239","title":"Experiments on Parallel Training of Deep Neural Network using Model Averaging","created":"2015-07-05","cited_by_count":14},{"id":"2306.08590","title":"Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning","created":"2023-06-14","cited_by_count":10},{"id":"2006.06377","title":"STL-SGD: Speeding Up Local SGD with Stagewise Communication Period","created":"2020-06-11","cited_by_count":8},{"id":"2111.04004","title":"Exponential escape efficiency of SGD from sharp minima in non-stationary regime","created":"2021-11-07","cited_by_count":5},{"id":"2210.16400","title":"Flatter, faster: scaling momentum for optimal speedup of SGD","created":"2022-10-28","cited_by_count":5}],"next_cursor":null}