{"list":"other","total":63,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"1608.03983","title":"SGDR: Stochastic Gradient Descent with Warm Restarts","created":"2016-08-13","cited_by_count":5045},{"id":"1502.01852","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","created":"2015-02-06","cited_by_count":3611},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"2004.10964","title":"Don't Stop Pretraining: Adapt Language Models to Domains and Tasks","created":"2020-04-23","cited_by_count":1565},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"1904.09237","title":"On the Convergence of Adam and Beyond","created":"2019-04-19","cited_by_count":841},{"id":"1312.6120","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","created":"2013-12-20","cited_by_count":763},{"id":"1309.5549","title":"Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming","created":"2013-09-22","cited_by_count":724},{"id":"1812.07956","title":"On Lazy Training in Differentiable Programming","created":"2018-12-19","cited_by_count":675},{"id":"1506.01186","title":"Cyclical Learning Rates for Training Neural Networks","created":"2015-06-03","cited_by_count":629},{"id":"1606.04474","title":"Learning to learn by gradient descent by gradient descent","created":"2016-06-14","cited_by_count":620},{"id":"1608.04636","title":"Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-\\L{}ojasiewicz Condition","created":"2016-08-16","cited_by_count":539},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"1903.08855","title":"Linguistic Knowledge and Transferability of Contextual Representations","created":"2019-03-21","cited_by_count":383},{"id":"1711.09846","title":"Population Based Training of Neural Networks","created":"2017-11-27","cited_by_count":367},{"id":"1502.03492","title":"Gradient-based Hyperparameter Optimization through Reversible Learning","created":"2015-02-11","cited_by_count":360},{"id":"1907.08610","title":"Lookahead Optimizer: k steps forward, 1 step back","created":"2019-07-19","cited_by_count":336},{"id":"1911.02590","title":"Optimizing Millions of Hyperparameters by Implicit Differentiation","created":"2019-11-06","cited_by_count":304},{"id":"1703.00887","title":"How to Escape Saddle Points Efficiently","created":"2017-03-02","cited_by_count":285},{"id":"2010.07468","title":"AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients","created":"2020-10-15","cited_by_count":283},{"id":"1606.05340","title":"Exponential expressivity in deep neural networks through transient chaos","created":"2016-06-16","cited_by_count":261},{"id":"1605.07110","title":"Deep Learning without Poor Local Minima","created":"2016-05-23","cited_by_count":234},{"id":"1602.02355","title":"Hyperparameter optimization with approximate gradient","created":"2016-02-07","cited_by_count":233},{"id":"1901.09321","title":"Fixup Initialization: Residual Learning Without Normalization","created":"2019-01-27","cited_by_count":198},{"id":"1606.01885","title":"Learning to Optimize","created":"2016-06-06","cited_by_count":195},{"id":"1412.6544","title":"Qualitatively characterizing neural network optimization problems","created":"2014-12-19","cited_by_count":188},{"id":"1906.02717","title":"Adaptive Gradient-Based Meta-Learning Methods","created":"2019-06-06","cited_by_count":183},{"id":"2003.02395","title":"A Simple Convergence Proof of Adam and Adagrad","created":"2020-03-05","cited_by_count":171},{"id":"1711.04735","title":"Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice","created":"2017-11-13","cited_by_count":132},{"id":"1704.00708","title":"No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis","created":"2017-04-03","cited_by_count":129},{"id":"1703.04782","title":"Online Learning Rate Adaptation with Hypergradient Descent","created":"2017-03-14","cited_by_count":118},{"id":"1808.05671","title":"On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization","created":"2018-08-16","cited_by_count":91},{"id":"1511.07147","title":"A PAC Approach to Application-Specific Algorithm Selection","created":"2015-11-23","cited_by_count":58},{"id":"1903.10399","title":"Learning-to-Learn Stochastic Gradient Descent with Biased Regularization","created":"2019-03-25","cited_by_count":49},{"id":"2406.09405","title":"Why Warmup the Learning Rate? Underlying Mechanisms and Improvements","created":"2024-06-13","cited_by_count":46},{"id":"1711.03091","title":"Dispersion for Data-Driven Algorithm Design, Online Learning, and Private Optimization","created":"2017-11-08","cited_by_count":37},{"id":"2402.15926","title":"Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiency","created":"2024-02-24","cited_by_count":35},{"id":"1611.04535","title":"Learning-Theoretic Foundations of Algorithm Configuration for Combinatorial Partitioning Problems","created":"2016-11-14","cited_by_count":30},{"id":"2208.01857","title":"The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift","created":"2022-08-03","cited_by_count":26},{"id":"2206.07886","title":"Generalization Bounds for Data-Driven Numerical Linear Algebra","created":"2022-06-16","cited_by_count":24},{"id":"2107.01739","title":"KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks","created":"2021-07-04","cited_by_count":20},{"id":"1804.10587","title":"An improvement of the convergence proof of the ADAM-Optimizer","created":"2018-04-27","cited_by_count":19},{"id":"2108.08770","title":"Learning-to-learn non-convex piecewise-Lipschitz functions","created":"2021-08-19","cited_by_count":15},{"id":"2207.10199","title":"Provably tuning the ElasticNet across instances","created":"2022-07-20","cited_by_count":14},{"id":"2003.02981","title":"Learning Complexity of Simulated Annealing","created":"2020-03-06","cited_by_count":12},{"id":"2406.00819","title":"Sample Complexity of Posted Pricing for a Single Item","created":"2024-06-02","cited_by_count":11},{"id":"2409.04367","title":"Algorithm Configuration for Structured Pfaffian Settings","created":"2024-09-06","cited_by_count":11},{"id":"2102.10734","title":"Provable Super-Convergence with a Large Cyclical Learning Rate","created":"2021-02-22","cited_by_count":10},{"id":"2309.00203","title":"Generalization Bound and Learning Methods for Data-Driven Projections in Linear Programming","created":"2023-09-01","cited_by_count":10},{"id":"2210.02070","title":"Non-Convergence and Limit Cycles in the Adam optimizer","created":"2022-10-05","cited_by_count":9},{"id":"2501.13734","title":"Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function","created":"2025-01-23","cited_by_count":9},{"id":"2204.03569","title":"Accelerating ERM for data-driven algorithm design using output-sensitive techniques","created":"2022-04-07","cited_by_count":8},{"id":"2403.06871","title":"On the Generalization Ability of Unsupervised Pretraining","created":"2024-03-11","cited_by_count":8},{"id":"2501.02926","title":"Offline-to-online hyperparameter transfer for stochastic bandits","created":"2025-01-06","cited_by_count":7},{"id":"2504.04105","title":"Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes","created":"2025-04-05","cited_by_count":7},{"id":"1805.08095","title":"Small steps and giant leaps: Minimal Newton solvers for Deep Learning","created":"2018-05-21","cited_by_count":6},{"id":"2208.03835","title":"On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks","created":"2022-08-07","cited_by_count":6},{"id":"2405.13992","title":"Learning Cut Generating Functions for Integer Programming","created":"2024-05-22","cited_by_count":5},{"id":"2212.13551","title":"On the Lower Bound of Minimizing Polyak-{\\L}ojasiewicz Functions","created":"2022-12-27","cited_by_count":3},{"id":"2101.00218","title":"An iterative K-FAC algorithm for Deep Learning","created":"2021-01-01","cited_by_count":2},{"id":"2104.08410","title":"Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models","created":"2021-04-17","cited_by_count":2},{"id":"2412.13473","title":"Learning complexity of gradient descent and conjugate gradient algorithms","created":"2024-12-18","cited_by_count":2}],"next_cursor":null}