{"list":"other","total":68,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1502.03167","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","created":"2015-02-11","cited_by_count":7005},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"2006.16236","title":"Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention","created":"2020-06-29","cited_by_count":2022},{"id":"2001.04451","title":"Reformer: The Efficient Transformer","created":"2020-01-13","cited_by_count":1668},{"id":"2009.14794","title":"Rethinking Attention with Performers","created":"2020-09-30","cited_by_count":1601},{"id":"1904.10509","title":"Generating Long Sequences with Sparse Transformers","created":"2019-04-23","cited_by_count":1546},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"2006.04768","title":"Linformer: Self-Attention with Linear Complexity","created":"2020-06-08","cited_by_count":1474},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"1807.03819","title":"Universal Transformers","created":"2018-07-10","cited_by_count":647},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1711.00489","title":"Don't Decay the Learning Rate, Increase the Batch Size","created":"2017-11-01","cited_by_count":320},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"1705.08741","title":"Train longer, generalize better: closing the generalization gap in large batch training of neural networks","created":"2017-05-24","cited_by_count":279},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"1806.00468","title":"Implicit Bias of Gradient Descent on Linear Convolutional Networks","created":"2018-06-01","cited_by_count":251},{"id":"1705.08292","title":"The Marginal Value of Adaptive Gradient Methods in Machine Learning","created":"2017-05-23","cited_by_count":250},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2110.15343","title":"Scatterbrain: Unifying Sparse and Low-rank Attention Approximation","created":"2021-10-28","cited_by_count":130},{"id":"1506.02617","title":"Path-SGD: Path-Normalized Optimization in Deep Neural Networks","created":"2015-06-08","cited_by_count":120},{"id":"2105.15134","title":"Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning","created":"2021-05-31","cited_by_count":119},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102},{"id":"2302.13214","title":"Fast Attention Requires Bounded Entries","created":"2023-02-26","cited_by_count":102},{"id":"1810.07896","title":"Solving Linear Programs in the Current Matrix Multiplication Time","created":"2018-10-18","cited_by_count":100},{"id":"1905.10961","title":"Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks","created":"2019-05-27","cited_by_count":76},{"id":"2112.00029","title":"Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models","created":"2021-11-30","cited_by_count":76},{"id":"2002.12915","title":"The Implicit and Explicit Regularization Effects of Dropout","created":"2020-02-28","cited_by_count":71},{"id":"1812.03981","title":"Theoretical Analysis of Auto Rate-Tuning by Batch Normalization","created":"2018-12-10","cited_by_count":65},{"id":"1803.01814","title":"Norm matters: efficient and accurate normalization schemes in deep networks","created":"2018-03-05","cited_by_count":62},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62},{"id":"2006.11648","title":"Training (Overparametrized) Neural Networks in Near-Linear Time","created":"2020-06-20","cited_by_count":58},{"id":"2302.02451","title":"KDEformer: Accelerating Transformers via Kernel Density Estimation","created":"2023-02-05","cited_by_count":58},{"id":"2004.04250","title":"An Improved Cutting Plane Method for Convex Optimization, Convex-Concave Games and its Applications","created":"2020-04-08","cited_by_count":57},{"id":"1702.08503","title":"SGD Learns the Conjugate Kernel Class of the Network","created":"2017-02-27","cited_by_count":54},{"id":"1905.04447","title":"Solving Empirical Risk Minimization in the Current Matrix Multiplication Time","created":"2019-05-11","cited_by_count":53},{"id":"2009.10217","title":"A Faster Interior Point Method for Semidefinite Programming","created":"2020-09-21","cited_by_count":53},{"id":"2208.02813","title":"Towards Understanding Mixture of Experts in Deep Learning","created":"2022-08-04","cited_by_count":52},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49},{"id":"2007.02876","title":"A Mathematical Theory of Attention","created":"2020-07-06","cited_by_count":46},{"id":"1704.08246","title":"Relative Error Tensor Low Rank Approximation","created":"2017-04-26","cited_by_count":45},{"id":"2004.07470","title":"Faster Dynamic Matrix Inverse for Faster LPs","created":"2020-04-16","cited_by_count":44},{"id":"2010.05315","title":"SMYRF: Efficient Attention using Asymmetric Clustering","created":"2020-10-11","cited_by_count":43},{"id":"1910.11957","title":"A Deterministic Linear Program Solver in Current Matrix Multiplication Time","created":"2019-10-25","cited_by_count":42},{"id":"1905.11675","title":"Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems","created":"2019-05-28","cited_by_count":40},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40},{"id":"2103.07601","title":"Approximating How Single Head Attention Learns","created":"2021-03-13","cited_by_count":38},{"id":"2211.06033","title":"A Faster Small Treewidth SDP Solver","created":"2022-11-11","cited_by_count":36},{"id":"2101.08208","title":"Solving SDP Faster: A Robust IPM Framework and Efficient Implementation","created":"2021-01-20","cited_by_count":33},{"id":"1712.09473","title":"Sketching for Kronecker Product Regression and P-splines","created":"2017-12-27","cited_by_count":29},{"id":"2011.05365","title":"A Nearly-Linear Time Algorithm for Linear Programs with Small Treewidth: A Multiscale Representation of Robust Central Path","created":"2020-11-10","cited_by_count":29},{"id":"2211.14407","title":"Faster Algorithm for Structured John Ellipsoid Computation","created":"2022-11-26","cited_by_count":28},{"id":"2208.04508","title":"Training Overparametrized Neural Networks in Sublinear Time","created":"2022-08-09","cited_by_count":22},{"id":"2210.12468","title":"Discrepancy Minimization in Input-Sparsity Time","created":"2022-10-22","cited_by_count":22},{"id":"2002.02304","title":"Solving Tall Dense Linear Programs in Nearly Linear Time","created":"2020-02-06","cited_by_count":21},{"id":"2102.12238","title":"Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm","created":"2021-02-24","cited_by_count":18},{"id":"1811.01442","title":"Towards a Zero-One Law for Column Subset Selection","created":"2018-11-04","cited_by_count":12},{"id":"2004.07986","title":"Average Case Column Subset Selection for Entrywise $\\ell_1$-Norm Loss","created":"2020-04-16","cited_by_count":12},{"id":"2106.02619","title":"Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions","created":"2021-06-04","cited_by_count":9},{"id":"2202.08489","title":"A Faster Interior-Point Method for Sum-of-Squares Optimization","created":"2022-02-17","cited_by_count":7},{"id":"2010.13888","title":"Unifying Matrix Data Structures: Simplifying and Speeding up Iterative Algorithms","created":"2020-10-26","cited_by_count":4},{"id":"1909.12441","title":"Total Least Squares Regression in Input Sparsity Time","created":"2019-09-27","cited_by_count":2}],"next_cursor":null}