{"list":"other","total":82,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2103.00020","title":"Learning Transferable Visual Models From Natural Language Supervision","created":"2021-02-26","cited_by_count":33744},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"1409.0473","title":"Neural Machine Translation by Jointly Learning to Align and Translate","created":"2014-09-01","cited_by_count":4548},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"2303.12712","title":"Sparks of Artificial General Intelligence: Early experiments with GPT-4","created":"2023-03-22","cited_by_count":2592},{"id":"2106.01345","title":"Decision Transformer: Reinforcement Learning via Sequence Modeling","created":"2021-06-02","cited_by_count":1546},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2205.06175","title":"A Generalist Agent","created":"2022-05-12","cited_by_count":808},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1812.07956","title":"On Lazy Training in Differentiable Programming","created":"2018-12-19","cited_by_count":675},{"id":"1810.02054","title":"Gradient Descent Provably Optimizes Over-parameterized Neural Networks","created":"2018-10-04","cited_by_count":659},{"id":"1811.03804","title":"Gradient Descent Finds Global Minima of Deep Neural Networks","created":"2018-11-09","cited_by_count":574},{"id":"1901.08584","title":"Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks","created":"2019-01-24","cited_by_count":566},{"id":"1804.06561","title":"A Mean Field View of the Landscape of Two-Layers Neural Networks","created":"2018-04-18","cited_by_count":466},{"id":"1805.09545","title":"On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport","created":"2018-05-24","cited_by_count":426},{"id":"1908.05355","title":"The generalization error of random features regression: Precise asymptotics and double descent curve","created":"2019-08-14","cited_by_count":413},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"1501.01571","title":"An Introduction to Matrix Concentration Inequalities","created":"2015-01-07","cited_by_count":397},{"id":"1811.04918","title":"Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers","created":"2018-11-12","cited_by_count":340},{"id":"1412.8690","title":"Breaking the Curse of Dimensionality with Convex Neural Networks","created":"2014-12-30","cited_by_count":331},{"id":"1906.06755","title":"Theoretical Limitations of Self-Attention in Neural Sequence Models","created":"2019-06-16","cited_by_count":288},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268},{"id":"1808.01204","title":"Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data","created":"2018-08-03","cited_by_count":261},{"id":"2301.13196","title":"Looped Transformers as Programmable Computers","created":"2023-01-30","cited_by_count":237},{"id":"2009.11848","title":"How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks","created":"2020-09-24","cited_by_count":222},{"id":"1606.01933","title":"A Decomposable Attention Model for Natural Language Inference","created":"2016-06-06","cited_by_count":215},{"id":"1808.00387","title":"Just Interpolate: Kernel \"Ridgeless\" Regression Can Generalize","created":"2018-08-01","cited_by_count":190},{"id":"2106.06981","title":"Thinking Like Transformers","created":"2021-06-13","cited_by_count":184},{"id":"1905.12173","title":"On the Inductive Bias of Neural Tangent Kernels","created":"2019-05-29","cited_by_count":177},{"id":"1902.04674","title":"Towards moderate overparameterization: global convergence guarantees for training shallow neural networks","created":"2019-02-12","cited_by_count":176},{"id":"1602.04474","title":"Generalization Properties of Learning with Random Features","created":"2016-02-14","cited_by_count":164},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161},{"id":"1904.12191","title":"Linearized two-layers neural networks in high dimension","created":"2019-04-27","cited_by_count":155},{"id":"2006.13409","title":"When Do Neural Networks Outperform Kernel Methods?","created":"2020-06-24","cited_by_count":150},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2006.14548","title":"Tensor Programs II: Neural Tangent Kernel for Any Architecture","created":"2020-06-25","cited_by_count":143},{"id":"2302.11055","title":"SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics","created":"2023-02-21","cited_by_count":143},{"id":"2202.08658","title":"The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks","created":"2022-02-17","cited_by_count":142},{"id":"1810.05369","title":"Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel","created":"2018-10-12","cited_by_count":131},{"id":"1901.03429","title":"On the Turing Completeness of Modern Neural Network Architectures","created":"2019-01-10","cited_by_count":129},{"id":"2006.10540","title":"Infinite attention: NNGP and NTK for deep attention networks","created":"2020-06-18","cited_by_count":121},{"id":"2101.10588","title":"Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration","created":"2021-01-26","cited_by_count":110},{"id":"2206.04301","title":"Unveiling Transformers with LEGO: a synthetic reasoning task","created":"2022-06-09","cited_by_count":106},{"id":"2105.11115","title":"Self-Attention Networks Can Process Bounded Hierarchical Languages","created":"2021-05-24","cited_by_count":105},{"id":"1904.00687","title":"On the Power and Limitations of Random Features for Understanding Neural Networks","created":"2019-04-01","cited_by_count":103},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102},{"id":"1909.08156","title":"Dynamics of Deep Neural Networks and Neural Tangent Hierarchy","created":"2019-09-18","cited_by_count":96},{"id":"1702.00887","title":"Structured Attention Networks","created":"2017-02-03","cited_by_count":95},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"1905.10337","title":"What Can ResNet Learn Efficiently, Going Beyond Kernels?","created":"2019-05-24","cited_by_count":91},{"id":"2102.13219","title":"Learning with invariances in random features and kernel models","created":"2021-02-25","cited_by_count":89},{"id":"2006.09286","title":"On the Computational Power of Transformers and its Implications in Sequence Modeling","created":"2020-06-16","cited_by_count":85},{"id":"1711.00501","title":"Learning One-hidden-layer Neural Networks with Landscape Design","created":"2017-11-01","cited_by_count":81},{"id":"1910.01619","title":"Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks","created":"2019-10-03","cited_by_count":79},{"id":"1909.11304","title":"Asymptotics of Wide Networks from Feynman Diagrams","created":"2019-09-25","cited_by_count":77},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"1911.00809","title":"Enhanced Convolutional Neural Tangent Kernels","created":"2019-11-03","cited_by_count":76},{"id":"2204.07697","title":"Theory of Graph Neural Networks: Representation and Learning","created":"2022-04-16","cited_by_count":61},{"id":"2002.06262","title":"Why Do Deep Residual Networks Generalize Better than Deep Feedforward Networks? -- A Neural Tangent Kernel Perspective","created":"2020-02-14","cited_by_count":60},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58},{"id":"1702.08503","title":"SGD Learns the Conjugate Kernel Class of the Network","created":"2017-02-27","cited_by_count":54},{"id":"2305.09828","title":"Mimetic Initialization of Self-Attention Layers","created":"2023-05-16","cited_by_count":54},{"id":"2302.06461","title":"A Study on ReLU and Softmax in Transformer","created":"2023-02-13","cited_by_count":53},{"id":"2306.03435","title":"On the Role of Attention in Prompt-tuning","created":"2023-06-06","cited_by_count":50},{"id":"1904.04326","title":"A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics","created":"2019-04-08","cited_by_count":49},{"id":"2006.13436","title":"Towards Understanding Hierarchical Learning: Benefits of Neural Representations","created":"2020-06-24","cited_by_count":46},{"id":"1906.08039","title":"The Barron Space and the Flow-induced Function Spaces for Neural Network Models","created":"2019-06-18","cited_by_count":39},{"id":"2103.07601","title":"Approximating How Single Head Attention Learns","created":"2021-03-13","cited_by_count":38},{"id":"2102.10032","title":"Approximation and Learning with Deep Convolutional Models: a Kernel Perspective","created":"2021-02-19","cited_by_count":31},{"id":"2111.08308","title":"Learning with convolution and pooling operations in kernel methods","created":"2021-11-16","cited_by_count":21},{"id":"2009.10008","title":"Kernel-Based Smoothness Analysis of Residual Networks","created":"2020-09-21","cited_by_count":19},{"id":"2110.08084","title":"Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization","created":"2021-10-15","cited_by_count":16},{"id":"2206.03688","title":"Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials","created":"2022-06-08","cited_by_count":13},{"id":"1908.10292","title":"On the Multiple Descent of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernels","created":"2019-08-27","cited_by_count":7}],"next_cursor":null}