{"list":"other","total":55,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"1706.08498","title":"Spectrally-normalized margin bounds for neural networks","created":"2017-06-26","cited_by_count":686},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"1412.8690","title":"Breaking the Curse of Dimensionality with Convex Neural Networks","created":"2014-12-30","cited_by_count":331},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2305.15408","title":"Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective","created":"2023-05-24","cited_by_count":294},{"id":"1906.06755","title":"Theoretical Limitations of Self-Attention in Neural Sequence Models","created":"2019-06-16","cited_by_count":288},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"2402.12875","title":"Chain of Thought Empowers Transformers to Solve Inherently Serial Problems","created":"2024-02-20","cited_by_count":260},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216},{"id":"2207.04901","title":"Exploring Length Generalization in Large Language Models","created":"2022-07-11","cited_by_count":206},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2302.11055","title":"SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics","created":"2023-02-21","cited_by_count":143},{"id":"1805.12076","title":"Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks","created":"2018-05-30","cited_by_count":142},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133},{"id":"1810.05369","title":"Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel","created":"2018-10-12","cited_by_count":131},{"id":"1910.01635","title":"A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case","created":"2019-10-03","cited_by_count":114},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102},{"id":"2210.15651","title":"Learning Single-Index Models with Shallow Neural Networks","created":"2022-10-27","cited_by_count":92},{"id":"2409.12640","title":"Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries","created":"2024-09-19","cited_by_count":88},{"id":"2406.15786","title":"What Matters in Transformers? Not All Attention is Needed","created":"2024-06-22","cited_by_count":84},{"id":"2402.09268","title":"Transformers, parallel computation, and logarithmic depth","created":"2024-02-14","cited_by_count":79},{"id":"2306.00946","title":"Exposing Attention Glitches with Flip-Flop Language Modeling","created":"2023-06-01","cited_by_count":75},{"id":"1312.6026","title":"How to Construct Deep Recurrent Neural Networks","created":"2013-12-20","cited_by_count":64},{"id":"2402.18510","title":"RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval","created":"2024-02-28","cited_by_count":64},{"id":"2209.14863","title":"Neural Networks Efficiently Learn Low-Dimensional Representations with SGD","created":"2022-09-29","cited_by_count":63},{"id":"2410.08633","title":"Transformers Provably Solve Parity Efficiently with Chain of Thought","created":"2024-10-11","cited_by_count":50},{"id":"2310.13897","title":"Masked Hard-Attention Transformers Recognize Exactly the Star-Free Languages","created":"2023-10-21","cited_by_count":49},{"id":"2106.03764","title":"On the Expressive Power of Self-Attention Matrices","created":"2021-06-07","cited_by_count":47},{"id":"1910.12947","title":"On Generalization Bounds of a Family of Recurrent Neural Networks","created":"2019-10-28","cited_by_count":42},{"id":"2306.02010","title":"Memorization Capacity of Multi-Head Attention in Transformers","created":"2023-06-03","cited_by_count":40},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40},{"id":"2408.12186","title":"Transformers are Minimax Optimal Nonparametric In-Context Learners","created":"2024-08-22","cited_by_count":37},{"id":"2402.05738","title":"Implicit Bias and Fast Convergence Rates for Self-attention","created":"2024-02-08","cited_by_count":31},{"id":"2411.02544","title":"Pretrained transformer efficiently learns low-dimensional target functions in-context","created":"2024-11-04","cited_by_count":31},{"id":"2310.13088","title":"Sequence Length Independent Norm-Based Generalization Bounds for Transformers","created":"2023-10-19","cited_by_count":29},{"id":"2406.09347","title":"Separations in the Representational Capabilities of Transformers and Recurrent Architectures","created":"2024-06-13","cited_by_count":29},{"id":"2411.04551","title":"Measure-to-measure interpolation using Transformers","created":"2024-11-07","cited_by_count":29},{"id":"2406.06893","title":"Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot","created":"2024-06-11","cited_by_count":22},{"id":"2410.19034","title":"Mixture of Parrots: Experts improve memorization more than reasoning","created":"2024-10-24","cited_by_count":21},{"id":"2408.07254","title":"Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics","created":"2024-08-14","cited_by_count":15},{"id":"2407.16153","title":"On the Benefits of Rank in Attention Layers","created":"2024-07-23","cited_by_count":5},{"id":"2410.11500","title":"On Generalisation Error Bounds for Transformers","created":"2024-10-15","cited_by_count":4},{"id":"2402.08808","title":"Depth Separation in Norm-Bounded Infinite-Width Neural Networks","created":"2024-02-13","cited_by_count":3},{"id":"2409.12446","title":"Neural Networks Generalize on Low Complexity Data","created":"2024-09-19","cited_by_count":3}],"next_cursor":null}