{"list":"other","total":94,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"1409.0575","title":"ImageNet Large Scale Visual Recognition Challenge","created":"2014-09-01","cited_by_count":8498},{"id":"1608.06993","title":"Densely Connected Convolutional Networks","created":"2016-08-25","cited_by_count":6720},{"id":"1710.09412","title":"mixup: Beyond Empirical Risk Minimization","created":"2017-10-25","cited_by_count":5183},{"id":"2201.03545","title":"A ConvNet for the 2020s","created":"2022-01-10","cited_by_count":3579},{"id":"1903.12261","title":"Benchmarking Neural Network Robustness to Common Corruptions and Perturbations","created":"2019-03-28","cited_by_count":2877},{"id":"2202.05262","title":"Locating and Editing Factual Associations in GPT","created":"2022-02-10","cited_by_count":2691},{"id":"2004.07780","title":"Shortcut Learning in Deep Neural Networks","created":"2020-04-16","cited_by_count":1969},{"id":"1811.12231","title":"ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness","created":"2018-11-29","cited_by_count":1432},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1806.08734","title":"On the Spectral Bias of Neural Networks","created":"2018-06-22","cited_by_count":1341},{"id":"1706.05394","title":"A Closer Look at Memorization in Deep Networks","created":"2017-06-16","cited_by_count":942},{"id":"2301.05217","title":"Progress measures for grokking via mechanistic interpretability","created":"2023-01-12","cited_by_count":839},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2304.14997","title":"Towards Automated Circuit Discovery for Mechanistic Interpretability","created":"2023-04-28","cited_by_count":724},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2108.08810","title":"Do Vision Transformers See Like Convolutional Neural Networks?","created":"2021-08-19","cited_by_count":675},{"id":"1803.02324","title":"Annotation Artifacts in Natural Language Inference Data","created":"2018-03-06","cited_by_count":671},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"1711.00165","title":"Deep Neural Networks as Gaussian Processes","created":"2017-11-01","cited_by_count":622},{"id":"1706.08947","title":"Exploring Generalization in Deep Learning","created":"2017-06-27","cited_by_count":600},{"id":"2105.10497","title":"Intriguing Properties of Vision Transformers","created":"2021-05-21","cited_by_count":446},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"2204.02937","title":"Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations","created":"2022-04-06","cited_by_count":381},{"id":"1901.06523","title":"Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks","created":"2019-01-19","cited_by_count":368},{"id":"2006.07710","title":"The Pitfalls of Simplicity Bias in Neural Networks","created":"2020-06-13","cited_by_count":351},{"id":"1412.6614","title":"In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning","created":"2014-12-20","cited_by_count":342},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"1804.11271","title":"Gaussian Process Behaviour in Wide Deep Neural Networks","created":"2018-04-30","cited_by_count":317},{"id":"1908.08593","title":"Revealing the Dark Secrets of BERT","created":"2019-08-21","cited_by_count":301},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"2011.09468","title":"Gradient Starvation: A Learning Proclivity in Neural Networks","created":"2020-11-18","cited_by_count":267},{"id":"2103.14586","title":"Understanding Robustness of Transformers for Image Classification","created":"2021-03-26","cited_by_count":249},{"id":"1905.00397","title":"Fast AutoAugment","created":"2019-05-01","cited_by_count":243},{"id":"2105.07581","title":"Vision Transformers are Robust Learners","created":"2021-05-17","cited_by_count":207},{"id":"1905.11604","title":"SGD on Neural Networks Learns Functions of Increasing Complexity","created":"2019-05-28","cited_by_count":204},{"id":"2305.00586","title":"How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model","created":"2023-04-30","cited_by_count":202},{"id":"2201.09792","title":"Patches Are All You Need?","created":"2022-01-24","cited_by_count":201},{"id":"2111.05328","title":"Data Augmentation Can Improve Robustness","created":"2021-11-09","cited_by_count":194},{"id":"1802.08760","title":"Sensitivity and Generalization in Neural Networks: an Empirical Study","created":"2018-02-23","cited_by_count":189},{"id":"2207.08799","title":"Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit","created":"2022-07-18","cited_by_count":187},{"id":"1805.08522","title":"Deep learning generalizes because the parameter-function map is biased towards simple functions","created":"2018-05-22","cited_by_count":183},{"id":"2006.06657","title":"Directional convergence and alignment in deep learning","created":"2020-06-11","cited_by_count":178},{"id":"1905.12173","title":"On the Inductive Bias of Neural Tangent Kernels","created":"2019-05-29","cited_by_count":177},{"id":"1912.01198","title":"Towards Understanding the Spectral Bias of Deep Learning","created":"2019-12-03","cited_by_count":174},{"id":"1810.05148","title":"Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes","created":"2018-10-11","cited_by_count":167},{"id":"2104.02610","title":"On the Robustness of Vision Transformers to Adversarial Examples","created":"2021-03-31","cited_by_count":167},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154},{"id":"1910.12478","title":"Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes","created":"2019-10-28","cited_by_count":151},{"id":"2012.09839","title":"Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning","created":"2020-12-17","cited_by_count":150},{"id":"2010.15775","title":"Understanding the Failure Modes of Out-of-Distribution Generalization","created":"2020-10-29","cited_by_count":146},{"id":"1808.05587","title":"Deep Convolutional Networks as shallow Gaussian Processes","created":"2018-08-16","cited_by_count":138},{"id":"1906.00425","title":"The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies","created":"2019-06-02","cited_by_count":135},{"id":"2309.07311","title":"Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs","created":"2023-09-13","cited_by_count":124},{"id":"1904.09080","title":"Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process","created":"2019-04-19","cited_by_count":123},{"id":"2112.13492","title":"Vision Transformer for Small-Size Datasets","created":"2021-12-27","cited_by_count":122},{"id":"2006.10540","title":"Infinite attention: NNGP and NTK for deep attention networks","created":"2020-06-18","cited_by_count":121},{"id":"1803.01905","title":"Convergence of Gradient Descent on Separable Data","created":"2018-03-05","cited_by_count":110},{"id":"2205.01580","title":"Better plain ViT baselines for ImageNet-1k","created":"2022-05-03","cited_by_count":98},{"id":"2302.07011","title":"A Modern Look at the Relationship between Sharpness and Generalization","created":"2023-02-14","cited_by_count":92},{"id":"2006.08680","title":"Shape Matters: Understanding the Implicit Bias of the Noise Covariance","created":"2020-06-15","cited_by_count":88},{"id":"2208.12591","title":"On the Implicit Bias in Deep-Learning Algorithms","created":"2022-08-26","cited_by_count":86},{"id":"1803.07300","title":"Risk and parameter convergence of logistic regression","created":"2018-03-20","cited_by_count":85},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83},{"id":"2002.06622","title":"Robustness Verification for Transformers","created":"2020-02-16","cited_by_count":76},{"id":"2310.10616","title":"How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations","created":"2023-10-16","cited_by_count":75},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74},{"id":"2402.09963","title":"Why are Sensitive Functions Hard for Transformers?","created":"2024-02-15","cited_by_count":74},{"id":"2211.12316","title":"Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions","created":"2022-11-22","cited_by_count":70},{"id":"1907.10599","title":"A Fine-Grained Spectral Perspective on Neural Networks","created":"2019-07-24","cited_by_count":68},{"id":"2310.03016","title":"Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions","created":"2023-10-04","cited_by_count":63},{"id":"2105.02723","title":"Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet","created":"2021-05-06","cited_by_count":61},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58},{"id":"2103.17268","title":"Fast Certified Robust Training with Short Warmup","created":"2021-03-31","cited_by_count":46},{"id":"2210.07082","title":"Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data","created":"2022-10-13","cited_by_count":46},{"id":"2207.04036","title":"Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent","created":"2022-07-08","cited_by_count":39},{"id":"2203.09125","title":"Are Vision Transformers Robust to Spurious Correlations?","created":"2022-03-17","cited_by_count":36},{"id":"2310.13121","title":"Understanding Addition in Transformers","created":"2023-10-19","cited_by_count":34},{"id":"2107.00595","title":"Fast Margin Maximization via Dual Acceleration","created":"2021-07-01","cited_by_count":33},{"id":"2302.00456","title":"Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Maps","created":"2023-02-01","cited_by_count":33},{"id":"2402.05738","title":"Implicit Bias and Fast Convergence Rates for Self-attention","created":"2024-02-08","cited_by_count":31},{"id":"2302.00457","title":"Simplicity Bias in 1-Hidden Layer Neural Networks","created":"2023-02-01","cited_by_count":26},{"id":"2211.03495","title":"How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers","created":"2022-11-07","cited_by_count":21},{"id":"2310.18935","title":"Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data","created":"2023-10-29","cited_by_count":18},{"id":"2311.17400","title":"Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention","created":"2023-11-29","cited_by_count":10},{"id":"2310.06161","title":"Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness","created":"2023-10-09","cited_by_count":7},{"id":"2402.02969","title":"Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features","created":"2024-02-05","cited_by_count":5},{"id":"2110.05301","title":"On a Benefit of Mask Language Modeling: Robustness to Simplicity Bias","created":"2021-10-11","cited_by_count":4}],"next_cursor":null}