{"list":"other","total":81,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2201.11903","title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","created":"2022-01-28","cited_by_count":12459},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"1906.08237","title":"XLNet: Generalized Autoregressive Pretraining for Language Understanding","created":"2019-06-19","cited_by_count":2811},{"id":"1909.11942","title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations","created":"2019-09-26","cited_by_count":2790},{"id":"2009.14794","title":"Rethinking Attention with Performers","created":"2020-09-30","cited_by_count":1601},{"id":"2006.04768","title":"Linformer: Self-Attention with Linear Complexity","created":"2020-06-08","cited_by_count":1474},{"id":"2105.01601","title":"MLP-Mixer: An all-MLP Architecture for Vision","created":"2021-05-04","cited_by_count":1446},{"id":"2202.12837","title":"Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?","created":"2022-02-25","cited_by_count":1434},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1506.06724","title":"Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books","created":"2015-06-22","cited_by_count":1043},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"1706.08947","title":"Exploring Generalization in Deep Learning","created":"2017-06-27","cited_by_count":600},{"id":"1703.11008","title":"Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data","created":"2017-03-31","cited_by_count":533},{"id":"1902.09229","title":"A Theoretical Analysis of Contrastive Unsupervised Representation Learning","created":"2019-02-25","cited_by_count":530},{"id":"1912.02178","title":"Fantastic Generalization Measures and Where to Find Them","created":"2019-12-04","cited_by_count":513},{"id":"1703.04933","title":"Sharp Minima Can Generalize For Deep Nets","created":"2017-03-15","cited_by_count":466},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"2105.08050","title":"Pay Attention to MLPs","created":"2021-05-17","cited_by_count":368},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"2106.04156","title":"Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss","created":"2021-06-08","cited_by_count":313},{"id":"2002.09277","title":"Kernel and Rich Regimes in Overparametrized Models","created":"2020-02-20","cited_by_count":306},{"id":"1704.04289","title":"Stochastic Gradient Descent as Approximate Bayesian Inference","created":"2017-04-13","cited_by_count":300},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"2102.01293","title":"Scaling Laws for Transfer","created":"2021-02-02","cited_by_count":272},{"id":"1806.00468","title":"Implicit Bias of Gradient Descent on Linear Convolutional Networks","created":"2018-06-01","cited_by_count":251},{"id":"2002.09434","title":"Few-Shot Learning via Learning the Representation, Provably","created":"2020-02-21","cited_by_count":214},{"id":"1503.01243","title":"A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights","created":"2015-03-04","cited_by_count":206},{"id":"2005.00743","title":"Synthesizer: Rethinking Self-Attention in Transformer Models","created":"2020-05-02","cited_by_count":205},{"id":"1810.02032","title":"Gradient descent aligns the layers of deep linear networks","created":"2018-10-04","cited_by_count":182},{"id":"1806.00900","title":"Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced","created":"2018-06-04","cited_by_count":177},{"id":"2006.03236","title":"Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing","created":"2020-06-05","cited_by_count":162},{"id":"2012.09839","title":"Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning","created":"2020-12-17","cited_by_count":150},{"id":"1808.08949","title":"Dissecting Contextual Word Embeddings: Architecture and Representation","created":"2018-08-27","cited_by_count":146},{"id":"1511.06251","title":"Stochastic modified equations and adaptive stochastic gradient algorithms","created":"2015-11-19","cited_by_count":142},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"1904.09080","title":"Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process","created":"2019-04-19","cited_by_count":123},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121},{"id":"1811.01558","title":"Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations","created":"2018-11-05","cited_by_count":114},{"id":"2110.08529","title":"Sharpness-Aware Minimization Improves Language Model Generalization","created":"2021-10-16","cited_by_count":108},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107},{"id":"2109.10686","title":"Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers","created":"2021-09-22","cited_by_count":106},{"id":"2206.04301","title":"Unveiling Transformers with LEGO: a synthetic reasoning task","created":"2022-06-09","cited_by_count":106},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102},{"id":"2202.14037","title":"Understanding Contrastive Learning Requires Incorporating Inductive Biases","created":"2022-02-28","cited_by_count":96},{"id":"1911.12246","title":"Do Attention Heads in BERT Track Syntactic Dependencies?","created":"2019-11-27","cited_by_count":88},{"id":"2006.08680","title":"Shape Matters: Understanding the Implicit Bias of the Noise Covariance","created":"2020-06-15","cited_by_count":88},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83},{"id":"2106.09226","title":"Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning","created":"2021-06-17","cited_by_count":82},{"id":"1909.05122","title":"Implicit Regularization for Optimal Sparse Recovery","created":"2019-09-11","cited_by_count":80},{"id":"2104.07705","title":"How to Train BERT with an Academic Budget","created":"2021-04-15","cited_by_count":80},{"id":"1910.01619","title":"Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks","created":"2019-10-03","cited_by_count":79},{"id":"2010.04925","title":"Regularizing Neural Networks via Adversarial Model Perturbation","created":"2020-10-10","cited_by_count":78},{"id":"1906.10225","title":"Compound Probabilistic Context-Free Grammars for Grammar Induction","created":"2019-06-24","cited_by_count":76},{"id":"2010.02501","title":"A Unifying View on Implicit Bias in Training Linear Neural Networks","created":"2020-10-06","cited_by_count":75},{"id":"2010.03648","title":"A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks","created":"2020-10-07","cited_by_count":72},{"id":"1905.03684","title":"Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation","created":"2019-05-09","cited_by_count":71},{"id":"2002.12915","title":"The Implicit and Explicit Regularization Effects of Dropout","created":"2020-02-28","cited_by_count":71},{"id":"2102.09769","title":"On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent","created":"2021-02-19","cited_by_count":70},{"id":"1904.01517","title":"Convergence rates for the stochastic gradient descent method for non-convex objective functions","created":"2019-04-02","cited_by_count":66},{"id":"1910.04284","title":"Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin","created":"2019-10-09","cited_by_count":56},{"id":"1703.08570","title":"Stochastic Methods for Composite and Weakly Convex Optimization Problems","created":"2017-03-24","cited_by_count":48},{"id":"1905.12917","title":"Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks","created":"2019-05-30","cited_by_count":48},{"id":"2006.01095","title":"Emergence of Separable Manifolds in Deep Language Representations","created":"2020-06-01","cited_by_count":40},{"id":"2111.01754","title":"Meta-Learning to Improve Pre-Training","created":"2021-11-02","cited_by_count":27},{"id":"2104.05694","title":"On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies","created":"2021-04-12","cited_by_count":18},{"id":"2105.15021","title":"Neural Bi-Lexicalized PCFG Induction","created":"2021-05-31","cited_by_count":18},{"id":"1910.10844","title":"Diametrical Risk Minimization: Theory and Computations","created":"2019-10-24","cited_by_count":15},{"id":"2011.01418","title":"Meta-learning Transferable Representations with a Single Target Domain","created":"2020-11-03","cited_by_count":5}],"next_cursor":null}