{"list":"other","total":73,"rows":[{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"1912.01703","title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","created":"2019-12-03","cited_by_count":20305},{"id":"1703.03400","title":"Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks","created":"2017-03-09","cited_by_count":4897},{"id":"2004.05439","title":"Meta-Learning in Neural Networks: A Survey","created":"2020-04-11","cited_by_count":950},{"id":"1904.09237","title":"On the Convergence of Adam and Beyond","created":"2019-04-19","cited_by_count":841},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"1908.03265","title":"On the Variance of the Adaptive Learning Rate and Beyond","created":"2019-08-08","cited_by_count":772},{"id":"1810.02054","title":"Gradient Descent Provably Optimizes Over-parameterized Neural Networks","created":"2018-10-04","cited_by_count":659},{"id":"1904.11955","title":"On Exact Computation with an Infinitely Wide Neural Net","created":"2019-04-26","cited_by_count":573},{"id":"1901.08584","title":"Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks","created":"2019-01-24","cited_by_count":566},{"id":"2302.06675","title":"Symbolic Discovery of Optimization Algorithms","created":"2023-02-13","cited_by_count":434},{"id":"1905.11881","title":"Why gradient clipping accelerates training: A theoretical justification for adaptivity","created":"2019-05-28","cited_by_count":341},{"id":"1711.00489","title":"Don't Decay the Learning Rate, Increase the Batch Size","created":"2017-11-01","cited_by_count":320},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"1808.01204","title":"Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data","created":"2018-08-03","cited_by_count":261},{"id":"1810.03548","title":"Meta-Learning: A Survey","created":"2018-10-08","cited_by_count":256},{"id":"1905.13210","title":"Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks","created":"2019-05-30","cited_by_count":222},{"id":"1902.09843","title":"Adaptive Gradient Methods with Dynamic Bound of Learning Rate","created":"2019-02-26","cited_by_count":183},{"id":"1902.04674","title":"Towards moderate overparameterization: global convergence guarantees for training shallow neural networks","created":"2019-02-12","cited_by_count":176},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"1805.08114","title":"On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes","created":"2018-05-21","cited_by_count":156},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"1603.06160","title":"Stochastic Variance Reduction for Nonconvex Optimization","created":"2016-03-19","cited_by_count":145},{"id":"1603.05953","title":"Katyusha: The First Direct Acceleration of Stochastic Gradient Methods","created":"2016-03-18","cited_by_count":135},{"id":"1906.04688","title":"An Improved Analysis of Training Over-parameterized Deep Neural Networks","created":"2019-06-11","cited_by_count":121},{"id":"1909.12292","title":"Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks","created":"2019-09-26","cited_by_count":121},{"id":"1910.07454","title":"An Exponential Learning Rate Schedule for Deep Learning","created":"2019-10-16","cited_by_count":121},{"id":"1703.04782","title":"Online Learning Rate Adaptation with Hypergradient Descent","created":"2017-03-14","cited_by_count":118},{"id":"2301.07733","title":"Learning-Rate-Free Learning by D-Adaptation","created":"2023-01-18","cited_by_count":113},{"id":"2301.08028","title":"A Tutorial on Meta-Reinforcement Learning","created":"2023-01-19","cited_by_count":108},{"id":"1706.03175","title":"Recovery Guarantees for One-hidden-layer Neural Networks","created":"2017-06-10","cited_by_count":94},{"id":"1810.12065","title":"On the Convergence Rate of Training Recurrent Neural Networks","created":"2018-10-29","cited_by_count":82},{"id":"2006.15081","title":"On the Generalization Benefit of Noise in Stochastic Gradient Descent","created":"2020-06-26","cited_by_count":82},{"id":"1905.10961","title":"Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks","created":"2019-05-27","cited_by_count":76},{"id":"1904.12838","title":"The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least Squares","created":"2019-04-29","cited_by_count":68},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62},{"id":"1906.03593","title":"Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound","created":"2019-06-09","cited_by_count":59},{"id":"2006.11648","title":"Training (Overparametrized) Neural Networks in Near-Linear Time","created":"2020-06-20","cited_by_count":58},{"id":"1905.13277","title":"Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence","created":"2019-05-30","cited_by_count":54},{"id":"2304.10411","title":"Attention Scheme Inspired Softmax Regression","created":"2023-04-20","cited_by_count":49},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44},{"id":"2304.13276","title":"The Closeness of In-Context Learning and Weight Shifting for Softmax Regression","created":"2023-04-26","cited_by_count":41},{"id":"1905.11675","title":"Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems","created":"2019-05-28","cited_by_count":40},{"id":"2102.05375","title":"Strength of Minibatch Noise in SGD","created":"2021-02-10","cited_by_count":39},{"id":"2002.06668","title":"Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality","created":"2020-02-16","cited_by_count":36},{"id":"2110.04622","title":"Does Preprocessing Help Training Over-parameterized Neural Networks?","created":"2021-10-09","cited_by_count":32},{"id":"2307.02419","title":"In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick","created":"2023-07-05","cited_by_count":32},{"id":"2306.04933","title":"InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding","created":"2023-06-08","cited_by_count":30},{"id":"2309.07418","title":"A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time","created":"2023-09-14","cited_by_count":30},{"id":"2211.14227","title":"Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing","created":"2022-11-25","cited_by_count":29},{"id":"2009.09829","title":"Generalized Leverage Score Sampling for Neural Networks","created":"2020-09-21","cited_by_count":26},{"id":"2308.12247","title":"How to Protect Copyright Data in Optimization of Large Language Models?","created":"2023-08-23","cited_by_count":26},{"id":"1810.00143","title":"AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods","created":"2018-09-29","cited_by_count":24},{"id":"2308.10502","title":"GradientCoin: A Peer-to-Peer Decentralized Large Language Models","created":"2023-08-21","cited_by_count":23},{"id":"1901.11150","title":"Memory-Efficient Adaptive Optimization","created":"2019-01-30","cited_by_count":19},{"id":"2208.03635","title":"Federated Adversarial Learning: A Framework with Convergence Analysis","created":"2022-08-07","cited_by_count":17},{"id":"2004.13146","title":"The Impact of the Mini-batch Size on the Variance of Gradients in Stochastic Gradient Descent","created":"2020-04-27","cited_by_count":16},{"id":"2007.05105","title":"AdaScale SGD: A User-Friendly Algorithm for Distributed Training","created":"2020-07-09","cited_by_count":16},{"id":"2206.12802","title":"Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis","created":"2022-06-26","cited_by_count":15},{"id":"1711.03440","title":"Learning Non-overlapping Convolutional Neural Networks with Multiple Kernels","created":"2017-11-08","cited_by_count":14},{"id":"2105.10762","title":"AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly","created":"2021-05-22","cited_by_count":14},{"id":"2210.02515","title":"Learning with Limited Samples -- Meta-Learning and Applications to Communication Systems","created":"2022-10-03","cited_by_count":13},{"id":"1908.06477","title":"Demystifying Learning Rate Policies for High Accuracy Training of Deep Neural Networks","created":"2019-08-18","cited_by_count":12},{"id":"2309.13482","title":"A Unified Scheme of ResNet and Softmax","created":"2023-09-23","cited_by_count":12},{"id":"2310.03331","title":"Fine-tune Language Models to Approximate Unbiased In-context Learning","created":"2023-10-05","cited_by_count":12},{"id":"2103.12682","title":"How to decay your learning rate","created":"2021-03-23","cited_by_count":11},{"id":"2209.12577","title":"Meta-Learning a Cross-lingual Manifold for Semantic Parsing","created":"2022-09-26","cited_by_count":11},{"id":"2307.06565","title":"Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification","created":"2023-07-13","cited_by_count":11},{"id":"2306.12700","title":"Accelerated Training via Incrementally Growing Neural Networks using Variance Transfer and Learning Rate Adaptation","created":"2023-06-22","cited_by_count":8},{"id":"2202.00145","title":"Step-size Adaptation Using Exponentiated Gradient Updates","created":"2022-01-31","cited_by_count":7},{"id":"2305.08001","title":"Structured and Fast Optimization: The Kronecker SGD Algorithm","created":"2023-05-13","cited_by_count":5},{"id":"2308.08847","title":"META-SELD: Meta-Learning for Fast Adaptation to the new environment in Sound Event Localization and Detection","created":"2023-08-17","cited_by_count":2}],"next_cursor":null}