{"list":"other","total":59,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"1906.08237","title":"XLNet: Generalized Autoregressive Pretraining for Language Understanding","created":"2019-06-19","cited_by_count":2811},{"id":"2007.14062","title":"Big Bird: Transformers for Longer Sequences","created":"2020-07-28","cited_by_count":1684},{"id":"2102.05095","title":"Is Space-Time Attention All You Need for Video Understanding?","created":"2021-02-09","cited_by_count":1620},{"id":"2106.01345","title":"Decision Transformer: Reinforcement Learning via Sequence Modeling","created":"2021-06-02","cited_by_count":1546},{"id":"1610.01145","title":"Error bounds for approximations with deep ReLU networks","created":"2016-10-03","cited_by_count":574},{"id":"1708.06633","title":"Nonparametric regression using deep neural networks with ReLU activation function","created":"2017-08-22","cited_by_count":433},{"id":"2104.11178","title":"VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text","created":"2021-04-22","cited_by_count":429},{"id":"2103.03404","title":"Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth","created":"2021-03-05","cited_by_count":419},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"1709.02540","title":"The Expressive Power of Neural Networks: A View from the Width","created":"2017-09-08","cited_by_count":320},{"id":"1910.06764","title":"Stabilizing Transformers for Reinforcement Learning","created":"2019-10-13","cited_by_count":270},{"id":"2006.04710","title":"The Lipschitz Constant of Self-Attention","created":"2020-06-08","cited_by_count":174},{"id":"1802.03620","title":"Optimal approximation of continuous functions by very deep ReLU networks","created":"2018-02-10","cited_by_count":166},{"id":"2001.03040","title":"Deep Network Approximation for Smooth Functions","created":"2020-01-09","cited_by_count":164},{"id":"1810.08033","title":"Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality","created":"2018-10-18","cited_by_count":162},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133},{"id":"1906.05497","title":"Deep Network Approximation Characterized by Number of Neurons","created":"2019-06-13","cited_by_count":122},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102},{"id":"2103.00502","title":"Optimal Approximation Rate of ReLU Networks in terms of Width and Depth","created":"2021-02-28","cited_by_count":96},{"id":"1902.07896","title":"Error bounds for approximations with deep ReLU neural networks in $W^{s,p}$ norms","created":"2019-02-21","cited_by_count":95},{"id":"2002.07028","title":"Low-Rank Bottleneck in Multi-head Attention Models","created":"2020-02-17","cited_by_count":89},{"id":"2007.15884","title":"The Kolmogorov-Arnold representation theorem revisited","created":"2020-07-31","cited_by_count":89},{"id":"1906.09477","title":"The phase diagram of approximation rates for deep neural networks","created":"2019-06-22","cited_by_count":88},{"id":"2101.12365","title":"Sharp Bounds on the Approximation Rates, Metric Entropy, and $n$-widths of Shallow Neural Networks","created":"2021-01-29","cited_by_count":86},{"id":"2006.04862","title":"$O(n)$ Connections are Expressive Enough: Universal Approximability of Sparse Transformers","created":"2020-06-08","cited_by_count":83},{"id":"2211.12316","title":"Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions","created":"2022-11-22","cited_by_count":70},{"id":"2010.14075","title":"Neural Network Approximation: Three Hidden Layers Are Enough","created":"2020-10-25","cited_by_count":63},{"id":"2205.13401","title":"Your Transformer May Not be as Powerful as You Expect","created":"2022-05-26","cited_by_count":55},{"id":"2211.14400","title":"Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces","created":"2022-11-25","cited_by_count":51},{"id":"2307.14023","title":"Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?","created":"2023-07-26","cited_by_count":47},{"id":"2007.02876","title":"A Mathematical Theory of Attention","created":"2020-07-06","cited_by_count":46},{"id":"2107.02397","title":"Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons","created":"2021-07-06","cited_by_count":41},{"id":"2306.02010","title":"Memorization Capacity of Multi-Head Attention in Transformers","created":"2023-06-03","cited_by_count":40},{"id":"2102.10911","title":"Elementary superexpressive activations","created":"2021-02-22","cited_by_count":36},{"id":"2110.03303","title":"Universal Approximation Under Constraints is Possible with Transformers","created":"2021-10-07","cited_by_count":32},{"id":"2305.18699","title":"Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Input","created":"2023-05-30","cited_by_count":32},{"id":"2411.06646","title":"Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data","created":"2024-11-11","cited_by_count":30},{"id":"2310.13088","title":"Sequence Length Independent Norm-Based Generalization Bounds for Transformers","created":"2023-10-19","cited_by_count":29},{"id":"2305.18475","title":"Approximation Rate of the Transformer Architecture for Sequence Modeling","created":"2023-05-29","cited_by_count":28},{"id":"2402.00522","title":"Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling","created":"2024-02-01","cited_by_count":25},{"id":"2402.14753","title":"Prompting a Pretrained Transformer Can Be a Universal Approximator","created":"2024-02-22","cited_by_count":25},{"id":"2306.08321","title":"Nonparametric regression using over-parameterized shallow ReLU neural networks","created":"2023-06-14","cited_by_count":24},{"id":"2406.06893","title":"Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot","created":"2024-06-11","cited_by_count":22},{"id":"2411.16525","title":"Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiency","created":"2024-11-25","cited_by_count":22},{"id":"2404.01601","title":"What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks","created":"2024-04-02","cited_by_count":19},{"id":"2105.03928","title":"Which transformer architecture fits my data? A vocabulary bottleneck in self-attention","created":"2021-05-09","cited_by_count":16},{"id":"2111.14574","title":"On the rate of convergence of a classifier based on a Transformer encoder","created":"2021-11-29","cited_by_count":16},{"id":"2404.02538","title":"Convergence Analysis of Flow Matching in Latent Space with Transformers","created":"2024-04-03","cited_by_count":16},{"id":"2409.00901","title":"On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks","created":"2024-09-02","cited_by_count":13},{"id":"2409.17677","title":"On the Optimal Memorization Capacity of Transformers","created":"2024-09-26","cited_by_count":10},{"id":"2407.09580","title":"Don't Fear Peculiar Activation Functions: EUAF and Beyond","created":"2024-07-12","cited_by_count":7},{"id":"2202.12166","title":"Attention Enables Zero Approximation Error","created":"2022-02-24","cited_by_count":4}],"next_cursor":null}