{"list":"other","total":84,"rows":[{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1704.04760","title":"In-Datacenter Performance Analysis of a Tensor Processing Unit","created":"2017-04-16","cited_by_count":618},{"id":"1802.04434","title":"signSGD: Compressed Optimisation for Non-Convex Problems","created":"2018-02-13","cited_by_count":571},{"id":"1608.04636","title":"Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-\\L{}ojasiewicz Condition","created":"2016-08-16","cited_by_count":539},{"id":"2302.06675","title":"Symbolic Discovery of Optimization Algorithms","created":"2023-02-13","cited_by_count":434},{"id":"2003.10422","title":"A Unified Theory of Decentralized SGD with Changing Topology and Local Updates","created":"2020-03-23","cited_by_count":367},{"id":"1402.4102","title":"Stochastic Gradient Hamiltonian Monte Carlo","created":"2014-02-17","cited_by_count":343},{"id":"1704.04289","title":"Stochastic Gradient Descent as Approximate Bayesian Inference","created":"2017-04-13","cited_by_count":300},{"id":"1703.00887","title":"How to Escape Saddle Points Efficiently","created":"2017-03-02","cited_by_count":285},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"1503.02101","title":"Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition","created":"2015-03-06","cited_by_count":243},{"id":"1905.13727","title":"PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization","created":"2019-05-31","cited_by_count":238},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"2301.11235","title":"Handbook of Convergence Theorems for (Stochastic) Gradient Methods","created":"2023-01-26","cited_by_count":163},{"id":"1511.06251","title":"Stochastic modified equations and adaptive stochastic gradient algorithms","created":"2015-11-19","cited_by_count":142},{"id":"2002.12410","title":"On Biased Compression for Distributed Learning","created":"2020-02-27","cited_by_count":140},{"id":"1705.07774","title":"Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients","created":"2017-05-22","cited_by_count":139},{"id":"2106.05203","title":"EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback","created":"2021-06-09","cited_by_count":133},{"id":"2006.04740","title":"The Heavy-Tail Phenomenon in SGD","created":"2020-06-08","cited_by_count":124},{"id":"1811.01558","title":"Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations","created":"2018-11-05","cited_by_count":114},{"id":"1901.09269","title":"Distributed Learning with Compressed Gradient Differences","created":"2019-01-26","cited_by_count":112},{"id":"2205.10287","title":"On the SDEs and Scaling Rules for Adaptive Gradient Algorithms","created":"2022-05-20","cited_by_count":104},{"id":"1905.10988","title":"Natural Compression for Distributed Deep Learning","created":"2019-05-27","cited_by_count":90},{"id":"2011.02828","title":"Local SGD: Unified Theory and New Efficient Methods","created":"2020-11-03","cited_by_count":88},{"id":"1905.11261","title":"A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent","created":"2019-05-27","cited_by_count":84},{"id":"2402.19449","title":"Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models","created":"2024-02-29","cited_by_count":80},{"id":"1609.05191","title":"Gradient Descent Learns Linear Dynamical Systems","created":"2016-09-16","cited_by_count":79},{"id":"2102.12470","title":"On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)","created":"2021-02-24","cited_by_count":73},{"id":"1903.06701","title":"Scaling Distributed Machine Learning with In-Network Aggregation","created":"2019-02-22","cited_by_count":65},{"id":"2206.07638","title":"Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays","created":"2022-06-15","cited_by_count":63},{"id":"1602.02666","title":"A Variational Analysis of Stochastic Gradient Algorithms","created":"2016-02-08","cited_by_count":57},{"id":"1806.06573","title":"Distributed learning with compressed gradients","created":"2018-06-18","cited_by_count":55},{"id":"2106.02969","title":"FedNL: Making Newton-Type Methods Applicable to Federated Learning","created":"2021-06-05","cited_by_count":52},{"id":"1905.12938","title":"Stochastic Sign Descent Methods: New Algorithms and Better Theory","created":"2019-05-30","cited_by_count":49},{"id":"2102.04396","title":"SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality","created":"2021-02-08","cited_by_count":35},{"id":"1801.00173","title":"Theory of Deep Learning III: explaining the non-overfitting puzzle","created":"2017-12-30","cited_by_count":32},{"id":"2305.15155","title":"Momentum Provably Improves Error Feedback!","created":"2023-05-24","cited_by_count":31},{"id":"1611.04831","title":"The Power of Normalization: Faster Evasion of Saddle Points","created":"2016-11-15","cited_by_count":29},{"id":"2006.07013","title":"A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimization","created":"2020-06-12","cited_by_count":28},{"id":"2103.17182","title":"Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization","created":"2021-03-31","cited_by_count":28},{"id":"2309.00079","title":"On the Implicit Bias of Adam","created":"2023-08-31","cited_by_count":27},{"id":"2102.07158","title":"Distributed Second Order Methods with Fast Rates and Compressed Communication","created":"2021-02-14","cited_by_count":25},{"id":"2310.01860","title":"High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise","created":"2023-10-03","cited_by_count":24},{"id":"2411.15958","title":"Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise","created":"2024-11-24","cited_by_count":23},{"id":"2301.08203","title":"An SDE for Modeling SAM: Theory and Insights","created":"2023-01-19","cited_by_count":22},{"id":"2306.02601","title":"Aiming towards the minimizers: fast convergence of SGD for overparametrized problems","created":"2023-06-05","cited_by_count":22},{"id":"1805.10111","title":"Double Quantization for Communication-Efficient Distributed Optimization","created":"2018-05-25","cited_by_count":19},{"id":"2107.09133","title":"The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion","created":"2021-07-19","cited_by_count":19},{"id":"2206.00737","title":"Walk for Learning: A Random Walk Approach for Federated Learning from Heterogeneous Data","created":"2022-06-01","cited_by_count":19},{"id":"1810.02565","title":"Continuous-time Models for Stochastic Optimization Algorithms","created":"2018-10-05","cited_by_count":17},{"id":"2205.04180","title":"EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization","created":"2022-05-09","cited_by_count":17},{"id":"2301.01542","title":"Federated Learning for Data Streams","created":"2023-01-04","cited_by_count":16},{"id":"2310.20452","title":"AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms","created":"2023-10-31","cited_by_count":16},{"id":"2111.01847","title":"Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learning","created":"2021-11-02","cited_by_count":15},{"id":"2311.05645","title":"EControl: Fast Distributed Optimization with Compression and Error Control","created":"2023-11-06","cited_by_count":15},{"id":"1702.06269","title":"Memory and Communication Efficient Distributed Stochastic Optimization with Minibatch-Prox","created":"2017-02-21","cited_by_count":12},{"id":"2206.03588","title":"Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregation","created":"2022-06-07","cited_by_count":11},{"id":"2211.10691","title":"Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States","created":"2022-11-19","cited_by_count":11},{"id":"2305.15850","title":"Stochastic Modified Equations and Dynamics of Dropout Algorithm","created":"2023-05-25","cited_by_count":11},{"id":"2305.18929","title":"Clip21: Error Feedback for Gradient Clipping","created":"2023-05-30","cited_by_count":11},{"id":"2403.04348","title":"LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression","created":"2024-03-07","cited_by_count":11},{"id":"2210.00690","title":"Taming Fat-Tailed (\"Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning","created":"2022-10-03","cited_by_count":10},{"id":"2305.09896","title":"Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compression","created":"2023-05-17","cited_by_count":9},{"id":"2410.12455","title":"Loss Landscape Characterization of Neural Networks without Over-Parametrization","created":"2024-10-16","cited_by_count":8},{"id":"2106.12261","title":"Learning Under Delayed Feedback: Implicitly Adapting to Gradient Delays","created":"2021-06-23","cited_by_count":7},{"id":"2302.07125","title":"Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent","created":"2023-02-14","cited_by_count":7},{"id":"2411.12135","title":"Exact Risk Curves of signSGD in High-Dimensions: Quantifying Preconditioning and Noise-Compression Effects","created":"2024-11-19","cited_by_count":7},{"id":"1803.08841","title":"The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory","created":"2018-03-23","cited_by_count":6},{"id":"2207.04922","title":"On uniform-in-time diffusion approximation for stochastic gradient descent","created":"2022-07-11","cited_by_count":6},{"id":"2310.17761","title":"Distributed Personalized Empirical Risk Minimization","created":"2023-10-26","cited_by_count":6},{"id":"2402.12508","title":"SDEs for Minimax Optimization","created":"2024-02-19","cited_by_count":6},{"id":"1711.09514","title":"Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms","created":"2017-11-27","cited_by_count":5},{"id":"2002.09304","title":"Stochastic Runge-Kutta methods and adaptive SGD-G2 stochastic gradient descent","created":"2020-02-20","cited_by_count":4},{"id":"2105.08037","title":"Adversarial Training for Gradient Descent: Analysis Through its Continuous-time Approximation","created":"2021-05-17","cited_by_count":4},{"id":"2106.04357","title":"Approximation to stochastic variance reduced gradient Langevin dynamics by stochastic delay differential equations","created":"2021-06-08","cited_by_count":4},{"id":"1803.05621","title":"Proximal SCOPE for Distributed Sparse Learning: Better Data Partition Implies Faster Convergence Rate","created":"2018-03-15","cited_by_count":3},{"id":"2312.15847","title":"Distributed Stochastic Optimization under Heavy-Tailed Noises","created":"2023-12-26","cited_by_count":3},{"id":"2003.03532","title":"Stochastic Modified Equations for Continuous Limit of Stochastic ADMM","created":"2020-03-07","cited_by_count":2},{"id":"2308.01358","title":"Compressed and distributed least-squares regression: convergence rates with applications to Federated Learning","created":"2023-08-02","cited_by_count":2},{"id":"2007.02723","title":"Weak error analysis for stochastic gradient descent optimization algorithms","created":"2020-07-03","cited_by_count":1},{"id":"2207.02750","title":"An SDE perspective on stochastic convex optimization","created":"2022-07-06","cited_by_count":1},{"id":"2209.05564","title":"Deep Relaxation of Controlled Stochastic Gradient Descent via Singular Perturbations","created":"2022-09-12","cited_by_count":1},{"id":"2210.11257","title":"A note on diffusion limits for stochastic gradient descent","created":"2022-10-20","cited_by_count":1},{"id":"2403.16775","title":"Stochastic Inertial Dynamics Via Time Scaling and Averaging","created":"2024-03-25","cited_by_count":1}],"next_cursor":null}