{"list":"other","total":53,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2108.07258","title":"On the Opportunities and Risks of Foundation Models","created":"2021-08-16","cited_by_count":4033},{"id":"2101.01169","title":"Transformers in Vision: A Survey","created":"2021-01-04","cited_by_count":1013},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2106.04554","title":"A Survey of Transformers","created":"2021-06-08","cited_by_count":401},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"1412.8690","title":"Breaking the Curse of Dimensionality with Convex Neural Networks","created":"2014-12-30","cited_by_count":331},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2305.15408","title":"Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective","created":"2023-05-24","cited_by_count":294},{"id":"1906.06755","title":"Theoretical Limitations of Self-Attention in Neural Sequence Models","created":"2019-06-16","cited_by_count":288},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268},{"id":"2402.12875","title":"Chain of Thought Empowers Transformers to Solve Inherently Serial Problems","created":"2024-02-20","cited_by_count":260},{"id":"2201.09873","title":"Transformers in Medical Imaging: A Survey","created":"2022-01-24","cited_by_count":245},{"id":"2301.13196","title":"Looped Transformers as Programmable Computers","created":"2023-01-30","cited_by_count":237},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216},{"id":"2212.10559","title":"Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers","created":"2022-12-20","cited_by_count":204},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133},{"id":"1901.03429","title":"On the Turing Completeness of Modern Neural Network Architectures","created":"2019-01-10","cited_by_count":129},{"id":"2305.19420","title":"What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization","created":"2023-05-30","cited_by_count":118},{"id":"2402.14735","title":"How Transformers Learn Causal Structure with Gradient Descent","created":"2024-02-22","cited_by_count":118},{"id":"2310.05249","title":"In-Context Convergence of Transformers","created":"2023-10-08","cited_by_count":115},{"id":"2105.11115","title":"Self-Attention Networks Can Process Bounded Hierarchical Languages","created":"2021-05-24","cited_by_count":105},{"id":"2309.05858","title":"Uncovering mesa-optimization algorithms in Transformers","created":"2023-09-11","cited_by_count":100},{"id":"2303.07971","title":"A Theory of Emergent In-Context Learning as Implicit Structure Induction","created":"2023-03-14","cited_by_count":86},{"id":"2306.04891","title":"In-Context Learning through the Bayesian Prism","created":"2023-06-08","cited_by_count":83},{"id":"2402.09268","title":"Transformers, parallel computation, and logarithmic depth","created":"2024-02-14","cited_by_count":79},{"id":"2310.10616","title":"How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations","created":"2023-10-16","cited_by_count":75},{"id":"2402.19442","title":"Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality","created":"2024-02-29","cited_by_count":74},{"id":"2402.18510","title":"RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval","created":"2024-02-28","cited_by_count":64},{"id":"2312.06528","title":"Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context","created":"2023-12-11","cited_by_count":63},{"id":"2402.08164","title":"On Limitations of the Transformer Architecture","created":"2024-02-13","cited_by_count":63},{"id":"2310.17086","title":"Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression","created":"2023-10-26","cited_by_count":49},{"id":"2106.03764","title":"On the Expressive Power of Self-Attention Matrices","created":"2021-06-07","cited_by_count":47},{"id":"2402.01258","title":"Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape","created":"2024-02-02","cited_by_count":46},{"id":"2412.02975","title":"Theoretical limitations of multi-layer Transformer","created":"2024-12-04","cited_by_count":45},{"id":"2408.01367","title":"Transformers are Universal In-context Learners","created":"2024-08-02","cited_by_count":43},{"id":"2408.10147","title":"In-Context Learning with Representations: Contextual Generalization of Trained Transformers","created":"2024-08-19","cited_by_count":38},{"id":"1910.06956","title":"Neural tangent kernels, transportation mappings, and universal approximation","created":"2019-10-15","cited_by_count":30},{"id":"2312.01429","title":"Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars","created":"2023-12-03","cited_by_count":30},{"id":"2402.14180","title":"Linear Transformers are Versatile In-Context Learners","created":"2024-02-21","cited_by_count":28},{"id":"1702.07028","title":"On the ability of neural nets to express distributions","created":"2017-02-22","cited_by_count":26},{"id":"2403.03183","title":"How Well Can Transformers Emulate In-context Newton's Method?","created":"2024-03-05","cited_by_count":25},{"id":"2402.04084","title":"Provably learning a multi-head attention layer","created":"2024-02-06","cited_by_count":21},{"id":"2310.08540","title":"Do pretrained Transformers Learn In-Context by Gradient Descent?","created":"2023-10-12","cited_by_count":18},{"id":"2405.16819","title":"Automatic Domain Adaptation by Transformers in In-Context Learning","created":"2024-05-27","cited_by_count":7},{"id":"2505.14808","title":"Out-of-Distribution Generalization of In-Context Learning: A Low-Dimensional Subspace Perspective","created":"2025-05-20","cited_by_count":6}],"next_cursor":null}