{"list":"other","total":54,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"1806.08734","title":"On the Spectral Bias of Neural Networks","created":"2018-06-22","cited_by_count":1341},{"id":"1602.07576","title":"Group Equivariant Convolutional Networks","created":"2016-02-24","cited_by_count":1165},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2301.05217","title":"Progress measures for grokking via mechanistic interpretability","created":"2023-01-12","cited_by_count":839},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"1711.00165","title":"Deep Neural Networks as Gaussian Processes","created":"2017-11-01","cited_by_count":622},{"id":"2010.14701","title":"Scaling Laws for Autoregressive Generative Modeling","created":"2020-10-28","cited_by_count":522},{"id":"2011.15091","title":"Inductive Biases for Deep Learning of Higher-Level Cognition","created":"2020-11-30","cited_by_count":275},{"id":"2205.10343","title":"Towards Understanding Grokking: An Effective Theory of Representation Learning","created":"2022-05-20","cited_by_count":230},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"2002.02561","title":"Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks","created":"2020-02-07","cited_by_count":200},{"id":"1912.01198","title":"Towards Understanding the Spectral Bias of Deep Learning","created":"2019-12-03","cited_by_count":174},{"id":"1810.05148","title":"Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes","created":"2018-10-11","cited_by_count":167},{"id":"2006.13198","title":"Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks","created":"2020-06-23","cited_by_count":166},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141},{"id":"2003.04560","title":"Frequency Bias in Neural Networks for Input of Non-Uniform Density","created":"2020-03-10","cited_by_count":140},{"id":"1906.00425","title":"The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies","created":"2019-06-02","cited_by_count":135},{"id":"2006.10540","title":"Infinite attention: NNGP and NTK for deep attention networks","created":"2020-06-18","cited_by_count":121},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105},{"id":"2310.01082","title":"Linear attention is (maybe) all you need (to understand transformer optimization)","created":"2023-10-02","cited_by_count":98},{"id":"2009.14397","title":"Deep Equals Shallow for ReLU Networks in Kernel Regimes","created":"2020-09-30","cited_by_count":85},{"id":"1909.11304","title":"Asymptotics of Wide Networks from Feynman Diagrams","created":"2019-09-25","cited_by_count":77},{"id":"2012.04030","title":"Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization","created":"2020-12-07","cited_by_count":71},{"id":"2211.12316","title":"Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions","created":"2022-11-22","cited_by_count":70},{"id":"1907.10599","title":"A Fine-Grained Spectral Perspective on Neural Networks","created":"2019-07-24","cited_by_count":68},{"id":"2003.02237","title":"Neural Kernels Without Tangents","created":"2020-03-04","cited_by_count":56},{"id":"2112.15383","title":"Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs","created":"2021-12-31","cited_by_count":53},{"id":"2006.15191","title":"Is SGD a Bayesian sampler? Well, almost","created":"2020-06-26","cited_by_count":50},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40},{"id":"2012.03636","title":"Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent","created":"2020-12-07","cited_by_count":37},{"id":"2110.14416","title":"Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs","created":"2021-10-27","cited_by_count":35},{"id":"2010.01590","title":"Deep kernel processes","created":"2020-10-04","cited_by_count":34},{"id":"2004.01190","title":"Predicting the outputs of finite deep neural networks trained with noisy gradients","created":"2020-04-02","cited_by_count":31},{"id":"2102.10032","title":"Approximation and Learning with Deep Convolutional Models: a Kernel Perspective","created":"2021-02-19","cited_by_count":31},{"id":"2303.14269","title":"The Exact Sample Complexity Gain from Invariances for Kernel Regression","created":"2023-03-24","cited_by_count":28},{"id":"2110.03922","title":"The Eigenlearning Framework: A Conservation Law Perspective on Kernel Regression and Wide Neural Networks","created":"2021-10-08","cited_by_count":26},{"id":"2106.02261","title":"Out-of-Distribution Generalization in Kernel Regression","created":"2021-06-04","cited_by_count":21},{"id":"2112.05589","title":"Unified field theoretical approach to deep and recurrent neuronal networks","created":"2021-12-10","cited_by_count":20},{"id":"2112.05611","title":"Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks","created":"2021-12-10","cited_by_count":17},{"id":"2203.09255","title":"On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels","created":"2022-03-17","cited_by_count":16},{"id":"2208.01003","title":"What Can Be Learnt With Wide Convolutional Neural Networks?","created":"2022-08-01","cited_by_count":16},{"id":"2106.07148","title":"On the Sample Complexity of Learning under Invariance and Geometric Stability","created":"2021-06-14","cited_by_count":13},{"id":"2205.15142","title":"Special Properties of Gradient Descent with Large Learning Rates","created":"2022-05-30","cited_by_count":10},{"id":"2202.13733","title":"On the Benefits of Large Learning Rates for Kernel Methods","created":"2022-02-28","cited_by_count":8},{"id":"2210.03275","title":"Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers","created":"2022-10-07","cited_by_count":2}],"next_cursor":null}