{"list":"other","total":58,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1807.03819","title":"Universal Transformers","created":"2018-07-10","cited_by_count":647},{"id":"2306.11644","title":"Textbooks Are All You Need","created":"2023-06-20","cited_by_count":569},{"id":"2309.05463","title":"Textbooks Are All You Need II: phi-1.5 technical report","created":"2023-09-11","cited_by_count":565},{"id":"2305.07759","title":"TinyStories: How Small Can Language Models Be and Still Speak Coherent English?","created":"2023-05-12","cited_by_count":449},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"1811.04918","title":"Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers","created":"2018-11-12","cited_by_count":340},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"1906.06755","title":"Theoretical Limitations of Self-Attention in Neural Sequence Models","created":"2019-06-16","cited_by_count":288},{"id":"2312.10794","title":"A mathematical perspective on Transformers","created":"2023-12-17","cited_by_count":164},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2302.11055","title":"SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics","created":"2023-02-21","cited_by_count":143},{"id":"2305.05465","title":"The emergence of clusters in self-attention dynamics","created":"2023-05-09","cited_by_count":140},{"id":"2106.16213","title":"Saturated Transformers are Constant-Depth Threshold Circuits","created":"2021-06-30","cited_by_count":139},{"id":"1705.09886","title":"Convergence Analysis of Two-layer Neural Networks with ReLU Activation","created":"2017-05-28","cited_by_count":137},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133},{"id":"2003.10409","title":"Online stochastic gradient descent on non-convex losses from high-dimensional inference","created":"2020-03-23","cited_by_count":119},{"id":"1906.02762","title":"Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View","created":"2019-06-06","cited_by_count":113},{"id":"2105.11115","title":"Self-Attention Networks Can Process Bounded Hierarchical Languages","created":"2021-05-24","cited_by_count":105},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102},{"id":"1706.03175","title":"Recovery Guarantees for One-hidden-layer Neural Networks","created":"2017-06-10","cited_by_count":94},{"id":"2204.06618","title":"Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity","created":"2022-04-13","cited_by_count":93},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"1711.00501","title":"Learning One-hidden-layer Neural Networks with Landscape Design","created":"2017-11-01","cited_by_count":81},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"1702.08503","title":"SGD Learns the Conjugate Kernel Class of the Network","created":"2017-02-27","cited_by_count":54},{"id":"1705.04591","title":"Learning ReLUs via Gradient Descent","created":"2017-05-10","cited_by_count":54},{"id":"1806.07808","title":"Learning One-hidden-layer ReLU Networks via Gradient Descent","created":"2018-06-20","cited_by_count":50},{"id":"2306.03435","title":"On the Role of Attention in Prompt-tuning","created":"2023-06-06","cited_by_count":50},{"id":"2006.12476","title":"Algorithms and SQ Lower Bounds for PAC Learning One-Hidden-Layer ReLU Networks","created":"2020-06-22","cited_by_count":49},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40},{"id":"1802.07895","title":"Learning Mixtures of Linear Regressions with Nearly Optimal Complexity","created":"2018-02-22","cited_by_count":37},{"id":"1611.10258","title":"Reliably Learning the ReLU in Polynomial Time","created":"2016-11-30","cited_by_count":36},{"id":"2004.13748","title":"Learning Polynomials of Few Relevant Dimensions","created":"2020-04-28","cited_by_count":34},{"id":"1811.01885","title":"Learning Two Layer Rectified Neural Networks in Polynomial Time","created":"2018-11-05","cited_by_count":31},{"id":"2310.13088","title":"Sequence Length Independent Norm-Based Generalization Bounds for Transformers","created":"2023-10-19","cited_by_count":29},{"id":"2012.07774","title":"Small Covers for Near-Zero Sets of Polynomials and Learning Latent Variable Models","created":"2020-12-14","cited_by_count":28},{"id":"2202.05258","title":"Hardness of Noise-Free Learning for Two-Hidden-Layer Neural Networks","created":"2022-02-10","cited_by_count":28},{"id":"2101.08303","title":"From Local Pseudorandom Generators to Hardness of Learning","created":"2021-01-20","cited_by_count":24},{"id":"1810.06793","title":"Learning Two-layer Neural Networks with Symmetric Inputs","created":"2018-10-16","cited_by_count":22},{"id":"1802.02547","title":"Learning One Convolutional Layer with Overlapping Patches","created":"2018-02-07","cited_by_count":21},{"id":"1805.02677","title":"Gradient Descent for One-Hidden-Layer Neural Networks: Polynomial Convergence and SQ Lower Bounds","created":"2018-05-07","cited_by_count":18},{"id":"1510.03528","title":"$\\ell_1$-regularized Neural Networks are Improperly Learnable in Polynomial Time","created":"2015-10-13","cited_by_count":16},{"id":"2304.10524","title":"Learning Narrow One-Hidden-Layer ReLU Networks","created":"2023-04-20","cited_by_count":14},{"id":"1802.07417","title":"Breaking the gridlock in Mixture-of-Experts: Consistent and Efficient Algorithms","created":"2018-02-21","cited_by_count":13},{"id":"2302.07426","title":"Computational Complexity of Learning Neural Networks: Smoothness and Degeneracy","created":"2023-02-15","cited_by_count":10},{"id":"2205.16003","title":"Learning (Very) Simple Generative Models Is Hard","created":"2022-05-31","cited_by_count":9},{"id":"1810.04133","title":"Learning One-hidden-layer Neural Networks under General Input Distributions","created":"2018-10-09","cited_by_count":7},{"id":"2205.03246","title":"What Makes A Good Fisherman? Linear Regression under Self-Selection Bias","created":"2022-05-06","cited_by_count":7},{"id":"2307.12496","title":"A faster and simpler algorithm for learning shallow networks","created":"2023-07-24","cited_by_count":7},{"id":"1906.02777","title":"Learning in Gated Neural Networks","created":"2019-06-06","cited_by_count":6},{"id":"2201.07206","title":"Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANs","created":"2022-01-18","cited_by_count":6},{"id":"2307.12840","title":"Efficiently Learning One-Hidden-Layer ReLU Networks via Schur Polynomials","created":"2023-07-24","cited_by_count":4}],"next_cursor":null}