{"list":"other","total":77,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2103.14030","title":"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows","created":"2021-03-25","cited_by_count":12139},{"id":"2012.12877","title":"Training data-efficient image transformers & distillation through attention","created":"2020-12-23","cited_by_count":4586},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"2106.01345","title":"Decision Transformer: Reinforcement Learning via Sequence Modeling","created":"2021-06-02","cited_by_count":1546},{"id":"2104.11227","title":"Multiscale Vision Transformers","created":"2021-04-22","cited_by_count":897},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"2008.08186","title":"Prevalence of Neural Collapse during the terminal phase of deep learning training","created":"2020-08-18","cited_by_count":719},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1810.02054","title":"Gradient Descent Provably Optimizes Over-parameterized Neural Networks","created":"2018-10-04","cited_by_count":659},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"2106.02039","title":"Offline Reinforcement Learning as One Big Sequence Modeling Problem","created":"2021-06-03","cited_by_count":644},{"id":"1906.11300","title":"Benign Overfitting in Linear Regression","created":"2019-06-26","cited_by_count":549},{"id":"2103.03404","title":"Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth","created":"2021-03-05","cited_by_count":419},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1705.04304","title":"A Deep Reinforced Model for Abstractive Summarization","created":"2017-05-11","cited_by_count":365},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"1705.09280","title":"Implicit Regularization in Matrix Factorization","created":"2017-05-25","cited_by_count":322},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"1703.03130","title":"A Structured Self-attentive Sentence Embedding","created":"2017-03-09","cited_by_count":318},{"id":"2002.09277","title":"Kernel and Rich Regimes in Overparametrized Models","created":"2020-02-20","cited_by_count":306},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"1906.06755","title":"Theoretical Limitations of Self-Attention in Neural Sequence Models","created":"2019-06-16","cited_by_count":288},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"1802.08246","title":"Characterizing Implicit Bias in Terms of Optimization Geometry","created":"2018-02-22","cited_by_count":266},{"id":"1808.01204","title":"Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data","created":"2018-08-03","cited_by_count":261},{"id":"2301.13196","title":"Looped Transformers as Programmable Computers","created":"2023-01-30","cited_by_count":237},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"1606.01933","title":"A Decomposable Attention Model for Natural Language Inference","created":"2016-06-06","cited_by_count":215},{"id":"1808.00387","title":"Just Interpolate: Kernel \"Ridgeless\" Regression Can Generalize","created":"2018-08-01","cited_by_count":190},{"id":"2006.06657","title":"Directional convergence and alignment in deep learning","created":"2020-06-11","cited_by_count":178},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"1601.06733","title":"Long Short-Term Memory-Networks for Machine Reading","created":"2016-01-25","cited_by_count":161},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"1904.09080","title":"Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process","created":"2019-04-19","cited_by_count":123},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121},{"id":"1803.01905","title":"Convergence of Gradient Descent on Separable Data","created":"2018-03-05","cited_by_count":110},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102},{"id":"1507.03566","title":"Low-rank Solutions of Linear Matrix Equations via Procrustes Flow","created":"2015-07-13","cited_by_count":97},{"id":"2005.08054","title":"Classification vs regression in overparameterized regimes: Does the loss function matter?","created":"2020-05-16","cited_by_count":97},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"1812.10004","title":"Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?","created":"2018-12-25","cited_by_count":88},{"id":"2006.08680","title":"Shape Matters: Understanding the Implicit Bias of the Noise Covariance","created":"2020-06-15","cited_by_count":88},{"id":"1803.07300","title":"Risk and parameter convergence of logistic regression","created":"2018-03-20","cited_by_count":85},{"id":"1909.05122","title":"Implicit Regularization for Optimal Sparse Recovery","created":"2019-09-11","cited_by_count":80},{"id":"2208.05512","title":"Imbalance Trouble: Revisiting Neural-Collapse Geometry","created":"2022-08-10","cited_by_count":80},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"2010.02501","title":"A Unifying View on Implicit Bias in Training Linear Neural Networks","created":"2020-10-06","cited_by_count":75},{"id":"2103.01550","title":"Label-Imbalanced and Group-Sensitive Classification under Overparameterization","created":"2021-03-02","cited_by_count":74},{"id":"2007.06738","title":"Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy","created":"2020-07-13","cited_by_count":71},{"id":"2202.06258","title":"Flowformer: Linearizing Transformers with Conservation Flows","created":"2022-02-13","cited_by_count":69},{"id":"2006.11226","title":"Gradient descent follows the regularization path for general losses","created":"2020-06-19","cited_by_count":59},{"id":"2201.12760","title":"Implicit Regularization Towards Rank Minimization in ReLU Networks","created":"2022-01-30","cited_by_count":53},{"id":"2306.07042","title":"Transformers learn through gradual rank increase","created":"2023-06-12","cited_by_count":52},{"id":"2306.03435","title":"On the Role of Attention in Prompt-tuning","created":"2023-06-06","cited_by_count":50},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47},{"id":"2012.06244","title":"The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks","created":"2020-12-11","cited_by_count":46},{"id":"1906.03830","title":"Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization","created":"2019-06-10","cited_by_count":40},{"id":"2010.09697","title":"Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent","created":"2020-10-19","cited_by_count":40},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40},{"id":"2107.00595","title":"Fast Margin Maximization via Dual Acceleration","created":"2021-07-01","cited_by_count":33},{"id":"2009.10670","title":"On the proliferation of support vectors in high dimensions","created":"2020-09-22","cited_by_count":32},{"id":"2108.04552","title":"The Benefits of Implicit Regularization from SGD in Least Squares Problems","created":"2021-08-10","cited_by_count":32},{"id":"2205.08078","title":"Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers","created":"2022-05-17","cited_by_count":30},{"id":"2011.09148","title":"Binary Classification of Gaussian Mixtures: Abundance of Support Vectors, Benign Overfitting and Regularization","created":"2020-11-18","cited_by_count":27},{"id":"2205.12808","title":"Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently","created":"2022-05-25","cited_by_count":26},{"id":"2207.11971","title":"Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformer","created":"2022-07-25","cited_by_count":21},{"id":"1906.03559","title":"The Implicit Bias of AdaGrad on Separable Data","created":"2019-06-09","cited_by_count":16},{"id":"2211.11052","title":"Convexifying Transformers: Improving optimization and understanding of transformer networks","created":"2022-11-20","cited_by_count":12},{"id":"2202.08371","title":"The Quarks of Attention","created":"2022-02-15","cited_by_count":7}],"next_cursor":null}