{"list":"other","total":56,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"1409.0473","title":"Neural Machine Translation by Jointly Learning to Align and Translate","created":"2014-09-01","cited_by_count":4548},{"id":"1708.07747","title":"Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms","created":"2017-08-25","cited_by_count":4349},{"id":"1606.04080","title":"Matching Networks for One Shot Learning","created":"2016-06-13","cited_by_count":2255},{"id":"1611.03530","title":"Understanding deep learning requires rethinking generalization","created":"2016-11-10","cited_by_count":1674},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"1905.09418","title":"Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned","created":"2019-05-23","cited_by_count":856},{"id":"1711.00165","title":"Deep Neural Networks as Gaussian Processes","created":"2017-11-01","cited_by_count":622},{"id":"2305.07759","title":"TinyStories: How Small Can Language Models Be and Still Speak Coherent English?","created":"2023-05-12","cited_by_count":449},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"1810.09538","title":"Pyro: Deep Universal Probabilistic Programming","created":"2018-10-18","cited_by_count":380},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"1703.03130","title":"A Structured Self-attentive Sentence Embedding","created":"2017-03-09","cited_by_count":318},{"id":"1804.11271","title":"Gaussian Process Behaviour in Wide Deep Neural Networks","created":"2018-04-30","cited_by_count":317},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"1906.06755","title":"Theoretical Limitations of Self-Attention in Neural Sequence Models","created":"2019-06-16","cited_by_count":288},{"id":"2310.17157","title":"Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time","created":"2023-10-26","cited_by_count":252},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"1606.01933","title":"A Decomposable Attention Model for Natural Language Inference","created":"2016-06-06","cited_by_count":215},{"id":"1701.02434","title":"A Conceptual Introduction to Hamiltonian Monte Carlo","created":"2017-01-10","cited_by_count":209},{"id":"1912.11554","title":"Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro","created":"2019-12-24","cited_by_count":191},{"id":"1810.10531","title":"A mathematical theory of semantic development in deep neural networks","created":"2018-10-23","cited_by_count":186},{"id":"2006.13198","title":"Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks","created":"2020-06-23","cited_by_count":166},{"id":"2312.10794","title":"A mathematical perspective on Transformers","created":"2023-12-17","cited_by_count":164},{"id":"1601.06733","title":"Long Short-Term Memory-Networks for Machine Reading","created":"2016-01-25","cited_by_count":161},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2006.10540","title":"Infinite attention: NNGP and NTK for deep attention networks","created":"2020-06-18","cited_by_count":121},{"id":"2305.19420","title":"What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization","created":"2023-05-30","cited_by_count":118},{"id":"2312.03002","title":"The mechanistic basis of data dependence and abrupt learning in an in-context classification task","created":"2023-12-03","cited_by_count":118},{"id":"2402.14735","title":"How Transformers Learn Causal Structure with Gradient Descent","created":"2024-02-22","cited_by_count":118},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102},{"id":"1702.00887","title":"Structured Attention Networks","created":"2017-02-03","cited_by_count":95},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"2310.10616","title":"How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations","created":"2023-10-16","cited_by_count":75},{"id":"2012.04030","title":"Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization","created":"2020-12-07","cited_by_count":71},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58},{"id":"1909.06576","title":"Torchmeta: A Meta-Learning library for PyTorch","created":"2019-09-14","cited_by_count":52},{"id":"2306.07042","title":"Transformers learn through gradual rank increase","created":"2023-06-12","cited_by_count":52},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47},{"id":"2304.03408","title":"Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks","created":"2023-04-06","cited_by_count":43},{"id":"2302.00375","title":"Bayes-optimal Learning of Deep Random Networks of Extensive-width","created":"2023-02-01","cited_by_count":41},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40},{"id":"2402.15607","title":"How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?","created":"2024-02-23","cited_by_count":35},{"id":"2205.08078","title":"Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers","created":"2022-05-17","cited_by_count":30},{"id":"2402.03902","title":"A phase transition between positional and semantic learning in a solvable model of dot-product attention","created":"2024-02-06","cited_by_count":30},{"id":"2212.14457","title":"Bayesian Interpolation with Deep Linear Networks","created":"2022-12-29","cited_by_count":21},{"id":"2304.07235","title":"Mapping of attention mechanisms to a generalized Potts model","created":"2023-04-14","cited_by_count":19},{"id":"2402.05173","title":"Towards Understanding Inductive Bias in Transformers: A View From Infinity","created":"2024-02-07","cited_by_count":13},{"id":"2210.17449","title":"Globally Gated Deep Linear Networks","created":"2022-10-31","cited_by_count":12},{"id":"2309.04522","title":"Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamics","created":"2023-09-08","cited_by_count":6}],"next_cursor":null}