{"list":"other","total":66,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2103.00020","title":"Learning Transferable Visual Models From Natural Language Supervision","created":"2021-02-26","cited_by_count":33744},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2012.12877","title":"Training data-efficient image transformers & distillation through attention","created":"2020-12-23","cited_by_count":4586},{"id":"1704.05426","title":"A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference","created":"2017-04-18","cited_by_count":2682},{"id":"2012.07421","title":"WILDS: A Benchmark of in-the-Wild Distribution Shifts","created":"2020-12-14","cited_by_count":1375},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"2103.03404","title":"Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth","created":"2021-03-05","cited_by_count":419},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1903.04561","title":"Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification","created":"2019-03-11","cited_by_count":404},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"2003.00307","title":"Loss landscapes and optimization in over-parameterized non-linear systems and neural networks","created":"2020-02-29","cited_by_count":233},{"id":"1905.13210","title":"Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks","created":"2019-05-30","cited_by_count":222},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"2006.06657","title":"Directional convergence and alignment in deep learning","created":"2020-06-11","cited_by_count":178},{"id":"2208.06677","title":"Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models","created":"2022-08-13","cited_by_count":174},{"id":"2306.00802","title":"Birth of a Transformer: A Memory Viewpoint","created":"2023-06-01","cited_by_count":161},{"id":"2012.09839","title":"Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning","created":"2020-12-17","cited_by_count":150},{"id":"2002.10542","title":"Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence","created":"2020-02-24","cited_by_count":137},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133},{"id":"2112.13492","title":"Vision Transformer for Small-Size Datasets","created":"2021-12-27","cited_by_count":122},{"id":"1903.09139","title":"Harmless interpolation of noisy data in regression","created":"2019-03-21","cited_by_count":121},{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114},{"id":"1803.01905","title":"Convergence of Gradient Descent on Separable Data","created":"2018-03-05","cited_by_count":110},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"2202.06526","title":"Benign Overfitting in Two-layer Convolutional Neural Networks","created":"2022-02-14","cited_by_count":91},{"id":"2208.12591","title":"On the Implicit Bias in Deep-Learning Algorithms","created":"2022-08-26","cited_by_count":86},{"id":"1803.07300","title":"Risk and parameter convergence of logistic regression","created":"2018-03-20","cited_by_count":85},{"id":"2006.04862","title":"$O(n)$ Connections are Expressive Enough: Universal Approximability of Sparse Transformers","created":"2020-06-08","cited_by_count":83},{"id":"2004.12019","title":"Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime","created":"2020-04-25","cited_by_count":81},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"1507.02030","title":"Beyond Convexity: Stochastic Quasi-Convex Optimization","created":"2015-07-08","cited_by_count":76},{"id":"2202.05928","title":"Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data","created":"2022-02-11","cited_by_count":74},{"id":"2002.07867","title":"Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology","created":"2020-02-18","cited_by_count":66},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52},{"id":"2306.03435","title":"On the Role of Attention in Prompt-tuning","created":"2023-06-06","cited_by_count":50},{"id":"2210.07082","title":"Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data","created":"2022-10-13","cited_by_count":46},{"id":"2012.05156","title":"Implicit Regularization in ReLU Networks with the Square Loss","created":"2020-12-09","cited_by_count":45},{"id":"2106.10865","title":"Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation","created":"2021-06-21","cited_by_count":42},{"id":"2010.09697","title":"Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent","created":"2020-10-19","cited_by_count":40},{"id":"2306.02010","title":"Memorization Capacity of Multi-Head Attention in Transformers","created":"2023-06-03","cited_by_count":40},{"id":"2402.04161","title":"Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains","created":"2024-02-06","cited_by_count":40},{"id":"2310.12680","title":"On the Optimization and Generalization of Multi-head Attention","created":"2023-10-19","cited_by_count":37},{"id":"2303.04145","title":"Benign Overfitting for Two-layer ReLU Convolutional Neural Networks","created":"2023-03-07","cited_by_count":34},{"id":"2107.00595","title":"Fast Margin Maximization via Dual Acceleration","created":"2021-07-01","cited_by_count":33},{"id":"2205.08078","title":"Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers","created":"2022-05-17","cited_by_count":30},{"id":"2303.01462","title":"Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization","created":"2023-03-02","cited_by_count":29},{"id":"2011.09148","title":"Binary Classification of Gaussian Mixtures: Abundance of Support Vectors, Benign Overfitting and Regularization","created":"2020-11-18","cited_by_count":27},{"id":"2305.07583","title":"MoMo: Momentum Models for Adaptive Learning Rates","created":"2023-05-12","cited_by_count":24},{"id":"2106.13792","title":"Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent","created":"2021-06-25","cited_by_count":21},{"id":"2302.09235","title":"Generalization and Stability of Interpolating Neural Networks with Minimal Width","created":"2023-02-18","cited_by_count":21},{"id":"2310.18935","title":"Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data","created":"2023-10-29","cited_by_count":18},{"id":"2211.11052","title":"Convexifying Transformers: Improving optimization and understanding of transformer networks","created":"2022-11-20","cited_by_count":12},{"id":"2202.08371","title":"The Quarks of Attention","created":"2022-02-15","cited_by_count":7},{"id":"2305.13471","title":"Fast Convergence in Learning Two-Layer Neural Networks with Separable Data","created":"2023-05-22","cited_by_count":3}],"next_cursor":null}