{"list":"other","total":85,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2103.00020","title":"Learning Transferable Visual Models From Natural Language Supervision","created":"2021-02-26","cited_by_count":33744},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"2201.11903","title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","created":"2022-01-28","cited_by_count":12459},{"id":"2107.03374","title":"Evaluating Large Language Models Trained on Code","created":"2021-07-07","cited_by_count":8320},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2012.12877","title":"Training data-efficient image transformers & distillation through attention","created":"2020-12-23","cited_by_count":4586},{"id":"1409.0473","title":"Neural Machine Translation by Jointly Learning to Align and Translate","created":"2014-09-01","cited_by_count":4548},{"id":"2102.12092","title":"Zero-Shot Text-to-Image Generation","created":"2021-02-24","cited_by_count":4288},{"id":"2108.07258","title":"On the Opportunities and Risks of Foundation Models","created":"2021-08-16","cited_by_count":4033},{"id":"2101.00190","title":"Prefix-Tuning: Optimizing Continuous Prompts for Generation","created":"2021-01-01","cited_by_count":3915},{"id":"2104.08691","title":"The Power of Scale for Parameter-Efficient Prompt Tuning","created":"2021-04-18","cited_by_count":3795},{"id":"2109.01652","title":"Finetuned Language Models Are Zero-Shot Learners","created":"2021-09-03","cited_by_count":3780},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"2303.03378","title":"PaLM-E: An Embodied Multimodal Language Model","created":"2023-03-06","cited_by_count":2071},{"id":"2106.01345","title":"Decision Transformer: Reinforcement Learning via Sequence Modeling","created":"2021-06-02","cited_by_count":1546},{"id":"1904.06690","title":"BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer","created":"2019-04-14","cited_by_count":964},{"id":"2205.06175","title":"A Generalist Agent","created":"2022-05-12","cited_by_count":808},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"1807.03819","title":"Universal Transformers","created":"2018-07-10","cited_by_count":647},{"id":"1811.03804","title":"Gradient Descent Finds Global Minima of Deep Neural Networks","created":"2018-11-09","cited_by_count":574},{"id":"1706.06978","title":"Deep Interest Network for Click-Through Rate Prediction","created":"2017-06-21","cited_by_count":544},{"id":"2103.03404","title":"Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth","created":"2021-03-05","cited_by_count":419},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1705.04304","title":"A Deep Reinforced Model for Abstractive Summarization","created":"2017-05-11","cited_by_count":365},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"1703.03130","title":"A Structured Self-attentive Sentence Embedding","created":"2017-03-09","cited_by_count":318},{"id":"2002.09277","title":"Kernel and Rich Regimes in Overparametrized Models","created":"2020-02-20","cited_by_count":306},{"id":"2305.15408","title":"Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective","created":"2023-05-24","cited_by_count":294},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"1802.08246","title":"Characterizing Implicit Bias in Terms of Optimization Geometry","created":"2018-02-22","cited_by_count":266},{"id":"2301.13196","title":"Looped Transformers as Programmable Computers","created":"2023-01-30","cited_by_count":237},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"1606.01933","title":"A Decomposable Attention Model for Natural Language Inference","created":"2016-06-06","cited_by_count":215},{"id":"2202.05607","title":"Online Decision Transformer","created":"2022-02-11","cited_by_count":196},{"id":"1810.02032","title":"Gradient descent aligns the layers of deep linear networks","created":"2018-10-04","cited_by_count":182},{"id":"2006.06657","title":"Directional convergence and alignment in deep learning","created":"2020-06-11","cited_by_count":178},{"id":"1902.04674","title":"Towards moderate overparameterization: global convergence guarantees for training shallow neural networks","created":"2019-02-12","cited_by_count":176},{"id":"2006.04710","title":"The Lipschitz Constant of Self-Attention","created":"2020-06-08","cited_by_count":174},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"1601.06733","title":"Long Short-Term Memory-Networks for Machine Reading","created":"2016-01-25","cited_by_count":161},{"id":"2104.10858","title":"All Tokens Matter: Token Labeling for Training Better Vision Transformers","created":"2021-04-22","cited_by_count":144},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2006.14548","title":"Tensor Programs II: Neural Tangent Kernel for Any Architecture","created":"2020-06-25","cited_by_count":143},{"id":"1905.06874","title":"Behavior Sequence Transformer for E-commerce Recommendation in Alibaba","created":"2019-05-15","cited_by_count":130},{"id":"1904.09080","title":"Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process","created":"2019-04-19","cited_by_count":123},{"id":"2006.10540","title":"Infinite attention: NNGP and NTK for deep attention networks","created":"2020-06-18","cited_by_count":121},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121},{"id":"1803.01905","title":"Convergence of Gradient Descent on Separable Data","created":"2018-03-05","cited_by_count":110},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107},{"id":"1804.09849","title":"The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation","created":"2018-04-26","cited_by_count":102},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"1812.10004","title":"Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?","created":"2018-12-25","cited_by_count":88},{"id":"2006.08680","title":"Shape Matters: Understanding the Implicit Bias of the Noise Covariance","created":"2020-06-15","cited_by_count":88},{"id":"1803.07300","title":"Risk and parameter convergence of logistic regression","created":"2018-03-20","cited_by_count":85},{"id":"1909.05122","title":"Implicit Regularization for Optimal Sparse Recovery","created":"2019-09-11","cited_by_count":80},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"2010.02501","title":"A Unifying View on Implicit Bias in Training Linear Neural Networks","created":"2020-10-06","cited_by_count":75},{"id":"2103.01550","title":"Label-Imbalanced and Group-Sensitive Classification under Overparameterization","created":"2021-03-02","cited_by_count":74},{"id":"2007.06738","title":"Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy","created":"2020-07-13","cited_by_count":71},{"id":"2006.11226","title":"Gradient descent follows the regularization path for general losses","created":"2020-06-19","cited_by_count":59},{"id":"2306.03435","title":"On the Role of Attention in Prompt-tuning","created":"2023-06-06","cited_by_count":50},{"id":"2012.06244","title":"The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks","created":"2020-12-11","cited_by_count":46},{"id":"2006.12467","title":"The Depth-to-Width Interplay in Self-Attention","created":"2020-06-22","cited_by_count":42},{"id":"1906.03830","title":"Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization","created":"2019-06-10","cited_by_count":40},{"id":"2103.07601","title":"Approximating How Single Head Attention Learns","created":"2021-03-13","cited_by_count":38},{"id":"1303.4172","title":"Margins, Shrinkage, and Boosting","created":"2013-03-18","cited_by_count":35},{"id":"2110.02732","title":"On Margin Maximization in Linear and ReLU Networks","created":"2021-10-06","cited_by_count":34},{"id":"2107.00595","title":"Fast Margin Maximization via Dual Acceleration","created":"2021-07-01","cited_by_count":33},{"id":"2108.04552","title":"The Benefits of Implicit Regularization from SGD in Least Squares Problems","created":"2021-08-10","cited_by_count":32},{"id":"2205.08078","title":"Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers","created":"2022-05-17","cited_by_count":30},{"id":"1806.00952","title":"Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization","created":"2018-06-04","cited_by_count":29},{"id":"2303.01462","title":"Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization","created":"2023-03-02","cited_by_count":29},{"id":"2305.18869","title":"Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning","created":"2023-05-30","cited_by_count":19},{"id":"1906.03559","title":"The Implicit Bias of AdaGrad on Separable Data","created":"2019-06-09","cited_by_count":16},{"id":"2211.11052","title":"Convexifying Transformers: Improving optimization and understanding of transformer networks","created":"2022-11-20","cited_by_count":12},{"id":"2209.07116","title":"On Generalization of Decentralized Learning with Separable Data","created":"2022-09-15","cited_by_count":8}],"next_cursor":null}