{"list":"other","total":62,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2103.00020","title":"Learning Transferable Visual Models From Natural Language Supervision","created":"2021-02-26","cited_by_count":33744},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2012.12877","title":"Training data-efficient image transformers & distillation through attention","created":"2020-12-23","cited_by_count":4586},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"1811.03804","title":"Gradient Descent Finds Global Minima of Deep Neural Networks","created":"2018-11-09","cited_by_count":574},{"id":"1901.08584","title":"Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks","created":"2019-01-24","cited_by_count":566},{"id":"2103.03404","title":"Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth","created":"2021-03-05","cited_by_count":419},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"1703.03130","title":"A Structured Self-attentive Sentence Embedding","created":"2017-03-09","cited_by_count":318},{"id":"1905.13210","title":"Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks","created":"2019-05-30","cited_by_count":222},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"1606.01933","title":"A Decomposable Attention Model for Natural Language Inference","created":"2016-06-06","cited_by_count":215},{"id":"1902.04674","title":"Towards moderate overparameterization: global convergence guarantees for training shallow neural networks","created":"2019-02-12","cited_by_count":176},{"id":"1601.06733","title":"Long Short-Term Memory-Networks for Machine Reading","created":"2016-01-25","cited_by_count":161},{"id":"2306.00802","title":"Birth of a Transformer: A Memory Viewpoint","created":"2023-06-01","cited_by_count":161},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133},{"id":"2010.01092","title":"On the linearity of large non-linear models: when and why the tangent kernel is constant","created":"2020-10-02","cited_by_count":129},{"id":"1909.12292","title":"Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks","created":"2019-09-26","cited_by_count":121},{"id":"2006.10540","title":"Infinite attention: NNGP and NTK for deep attention networks","created":"2020-06-18","cited_by_count":121},{"id":"2006.08157","title":"Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent","created":"2020-06-15","cited_by_count":119},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"2012.11654","title":"Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks","created":"2020-12-21","cited_by_count":87},{"id":"1803.07300","title":"Risk and parameter convergence of logistic regression","created":"2018-03-20","cited_by_count":85},{"id":"1911.12360","title":"How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?","created":"2019-11-27","cited_by_count":84},{"id":"2006.04862","title":"$O(n)$ Connections are Expressive Enough: Universal Approximability of Sparse Transformers","created":"2020-06-08","cited_by_count":83},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74},{"id":"2002.07867","title":"Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology","created":"2020-02-18","cited_by_count":66},{"id":"2306.03435","title":"On the Role of Attention in Prompt-tuning","created":"2023-06-06","cited_by_count":50},{"id":"2102.02410","title":"A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network","created":"2021-02-04","cited_by_count":47},{"id":"2106.03764","title":"On the Expressive Power of Self-Attention Matrices","created":"2021-06-07","cited_by_count":47},{"id":"2306.02010","title":"Memorization Capacity of Multi-Head Attention in Transformers","created":"2023-06-03","cited_by_count":40},{"id":"1303.4172","title":"Margins, Shrinkage, and Boosting","created":"2013-03-18","cited_by_count":35},{"id":"2006.01005","title":"The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks","created":"2020-06-01","cited_by_count":33},{"id":"2205.08078","title":"Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers","created":"2022-05-17","cited_by_count":30},{"id":"2007.00028","title":"Gradient Methods Never Overfit On Separable Data","created":"2020-06-30","cited_by_count":29},{"id":"2202.13441","title":"Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond","created":"2022-02-27","cited_by_count":29},{"id":"2107.12723","title":"Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel","created":"2021-07-27","cited_by_count":26},{"id":"2302.10034","title":"Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron","created":"2023-02-20","cited_by_count":25},{"id":"1905.09870","title":"Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems","created":"2019-05-23","cited_by_count":24},{"id":"2209.09298","title":"Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks","created":"2022-09-19","cited_by_count":21},{"id":"2302.09235","title":"Generalization and Stability of Interpolating Neural Networks with Minimal Width","created":"2023-02-18","cited_by_count":21},{"id":"2204.12446","title":"Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD","created":"2022-04-26","cited_by_count":19},{"id":"2101.04968","title":"Learning with Gradient Descent and Weakly Convex Losses","created":"2021-01-13","cited_by_count":16},{"id":"2211.11052","title":"Convexifying Transformers: Improving optimization and understanding of transformer networks","created":"2022-11-20","cited_by_count":12},{"id":"2305.19377","title":"Benign Overfitting in Deep Neural Networks under Lazy Training","created":"2023-05-30","cited_by_count":10},{"id":"2011.10036","title":"On the Dynamics of Training Attention Models","created":"2020-11-19","cited_by_count":8},{"id":"2202.08371","title":"The Quarks of Attention","created":"2022-02-15","cited_by_count":7},{"id":"2209.15106","title":"Restricted Strong Convexity of Deep Learning Models with Smooth Activations","created":"2022-09-29","cited_by_count":7}],"next_cursor":null}