{"list":"other","total":66,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1409.1556","title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","created":"2014-09-04","cited_by_count":15364},{"id":"2005.12872","title":"End-to-End Object Detection with Transformers","created":"2020-05-26","cited_by_count":7414},{"id":"1502.03167","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","created":"2015-02-11","cited_by_count":7005},{"id":"2104.14294","title":"Emerging Properties in Self-Supervised Vision Transformers","created":"2021-04-29","cited_by_count":6968},{"id":"1512.00567","title":"Rethinking the Inception Architecture for Computer Vision","created":"2015-12-02","cited_by_count":5475},{"id":"1710.09412","title":"mixup: Beyond Empirical Risk Minimization","created":"2017-10-25","cited_by_count":5183},{"id":"2012.12877","title":"Training data-efficient image transformers & distillation through attention","created":"2020-12-23","cited_by_count":4586},{"id":"1409.0473","title":"Neural Machine Translation by Jointly Learning to Align and Translate","created":"2014-09-01","cited_by_count":4548},{"id":"2108.07732","title":"Program Synthesis with Large Language Models","created":"2021-08-16","cited_by_count":3637},{"id":"1905.04899","title":"CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features","created":"2019-05-13","cited_by_count":2884},{"id":"1603.05027","title":"Identity Mappings in Deep Residual Networks","created":"2016-03-16","cited_by_count":1965},{"id":"2104.02057","title":"An Empirical Study of Training Self-Supervised Vision Transformers","created":"2021-04-05","cited_by_count":1616},{"id":"2106.01345","title":"Decision Transformer: Reinforcement Learning via Sequence Modeling","created":"2021-06-02","cited_by_count":1546},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1708.04896","title":"Random Erasing Data Augmentation","created":"2017-08-16","cited_by_count":1239},{"id":"1603.09382","title":"Deep Networks with Stochastic Depth","created":"2016-03-30","cited_by_count":979},{"id":"1805.09501","title":"AutoAugment: Learning Augmentation Policies from Data","created":"2018-05-24","cited_by_count":730},{"id":"2108.08810","title":"Do Vision Transformers See Like Convolutional Neural Networks?","created":"2021-08-19","cited_by_count":675},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"2106.02039","title":"Offline Reinforcement Learning as One Big Sequence Modeling Problem","created":"2021-06-03","cited_by_count":644},{"id":"2106.04803","title":"CoAtNet: Marrying Convolution and Attention for All Data Sizes","created":"2021-06-09","cited_by_count":569},{"id":"1906.05909","title":"Stand-Alone Self-Attention in Vision Models","created":"2019-06-13","cited_by_count":542},{"id":"2103.10697","title":"ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases","created":"2021-03-19","cited_by_count":484},{"id":"2105.10497","title":"Intriguing Properties of Vision Transformers","created":"2021-05-21","cited_by_count":446},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1904.09925","title":"Attention Augmented Convolutional Networks","created":"2019-04-22","cited_by_count":363},{"id":"1711.00489","title":"Don't Decay the Learning Rate, Increase the Batch Size","created":"2017-11-01","cited_by_count":320},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"2107.06263","title":"CMT: Convolutional Neural Networks Meet Vision Transformers","created":"2021-07-13","cited_by_count":310},{"id":"2002.09277","title":"Kernel and Rich Regimes in Overparametrized Models","created":"2020-02-20","cited_by_count":306},{"id":"1911.03584","title":"On the Relationship between Self-Attention and Convolutional Layers","created":"2019-11-08","cited_by_count":300},{"id":"2012.09816","title":"Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning","created":"2020-12-17","cited_by_count":297},{"id":"1705.08741","title":"Train longer, generalize better: closing the generalization gap in large batch training of neural networks","created":"2017-05-24","cited_by_count":279},{"id":"1912.01412","title":"Deep Learning for Symbolic Mathematics","created":"2019-12-02","cited_by_count":270},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"1806.00468","title":"Implicit Bias of Gradient Descent on Linear Convolutional Networks","created":"2018-06-01","cited_by_count":251},{"id":"1705.08292","title":"The Marginal Value of Adaptive Gradient Methods in Machine Learning","created":"2017-05-23","cited_by_count":250},{"id":"2103.14586","title":"Understanding Robustness of Transformers for Image Classification","created":"2021-03-26","cited_by_count":249},{"id":"2105.07581","title":"Vision Transformers are Robust Learners","created":"2021-05-17","cited_by_count":207},{"id":"2201.09792","title":"Patches Are All You Need?","created":"2022-01-24","cited_by_count":201},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144},{"id":"2006.10540","title":"Infinite attention: NNGP and NTK for deep attention networks","created":"2020-06-18","cited_by_count":121},{"id":"1506.02617","title":"Path-SGD: Path-Normalized Optimization in Deep Neural Networks","created":"2015-06-08","cited_by_count":120},{"id":"2105.15134","title":"Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning","created":"2021-05-31","cited_by_count":119},{"id":"2102.08602","title":"LambdaNetworks: Modeling Long-Range Interactions Without Attention","created":"2021-02-17","cited_by_count":107},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102},{"id":"1812.03981","title":"Theoretical Analysis of Auto Rate-Tuning by Batch Normalization","created":"2018-12-10","cited_by_count":65},{"id":"1803.01814","title":"Norm matters: efficient and accurate normalization schemes in deep networks","created":"2018-03-05","cited_by_count":62},{"id":"2105.02723","title":"Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet","created":"2021-05-06","cited_by_count":61},{"id":"1702.08503","title":"SGD Learns the Conjugate Kernel Class of the Network","created":"2017-02-27","cited_by_count":54},{"id":"2208.02813","title":"Towards Understanding Mixture of Experts in Deep Learning","created":"2022-08-04","cited_by_count":52},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49},{"id":"2007.02876","title":"A Mathematical Theory of Attention","created":"2020-07-06","cited_by_count":46},{"id":"2101.06223","title":"LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning","created":"2021-01-15","cited_by_count":46},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40},{"id":"2103.07601","title":"Approximating How Single Head Attention Learns","created":"2021-03-13","cited_by_count":38},{"id":"2007.06761","title":"Can neural networks acquire a structural bias from raw linguistic data?","created":"2020-07-14","cited_by_count":35},{"id":"2007.04596","title":"Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK","created":"2020-07-09","cited_by_count":18},{"id":"2102.12238","title":"Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm","created":"2021-02-24","cited_by_count":18},{"id":"2106.02619","title":"Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions","created":"2021-06-04","cited_by_count":9}],"next_cursor":null}