{"list":"other","total":88,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1606.08415","title":"Gaussian Error Linear Units (GELUs)","created":"2016-06-27","cited_by_count":3502},{"id":"2006.16236","title":"Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention","created":"2020-06-29","cited_by_count":2022},{"id":"2004.07780","title":"Shortcut Learning in Deep Neural Networks","created":"2020-04-16","cited_by_count":1969},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"1611.01578","title":"Neural Architecture Search with Reinforcement Learning","created":"2016-11-05","cited_by_count":1499},{"id":"2205.13504","title":"Are Transformers Effective for Time Series Forecasting?","created":"2022-05-26","cited_by_count":1424},{"id":"1806.08734","title":"On the Spectral Bias of Neural Networks","created":"2018-06-22","cited_by_count":1341},{"id":"2406.17557","title":"The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale","created":"2024-06-25","cited_by_count":1160},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"2102.04664","title":"CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation","created":"2021-02-09","cited_by_count":744},{"id":"1909.09436","title":"CodeSearchNet Challenge: Evaluating the State of Semantic Code Search","created":"2019-09-20","cited_by_count":654},{"id":"2305.18654","title":"Faith and Fate: Limits of Transformers on Compositionality","created":"2023-05-29","cited_by_count":503},{"id":"2102.11174","title":"Linear Transformers Are Secretly Fast Weight Programmers","created":"2021-02-22","cited_by_count":502},{"id":"1712.00559","title":"Progressive Neural Architecture Search","created":"2017-12-02","cited_by_count":487},{"id":"2305.07759","title":"TinyStories: How Small Can Language Models Be and Still Speak Coherent English?","created":"2023-05-12","cited_by_count":449},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1901.06523","title":"Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks","created":"2019-01-19","cited_by_count":368},{"id":"2305.19466","title":"The Impact of Positional Encoding on Length Generalization in Transformers","created":"2023-05-31","cited_by_count":327},{"id":"2011.15091","title":"Inductive Biases for Deep Learning of Higher-Level Cognition","created":"2020-11-30","cited_by_count":275},{"id":"2011.09468","title":"Gradient Starvation: A Learning Proclivity in Neural Networks","created":"2020-11-18","cited_by_count":267},{"id":"2301.05187","title":"WIRE: Wavelet Implicit Neural Representations","created":"2023-01-05","cited_by_count":220},{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216},{"id":"2207.04901","title":"Exploring Length Generalization in Large Language Models","created":"2022-07-11","cited_by_count":206},{"id":"1905.11604","title":"SGD on Neural Networks Learns Functions of Increasing Complexity","created":"2019-05-28","cited_by_count":204},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199},{"id":"1805.08522","title":"Deep learning generalizes because the parameter-function map is biased towards simple functions","created":"2018-05-22","cited_by_count":183},{"id":"2305.13169","title":"A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity","created":"2023-05-22","cited_by_count":182},{"id":"1710.05941","title":"Searching for Activation Functions","created":"2017-10-16","cited_by_count":169},{"id":"2109.08668","title":"Primer: Searching for Efficient Transformers for Language Modeling","created":"2021-09-17","cited_by_count":164},{"id":"2009.06489","title":"The Hardware Lottery","created":"2020-09-14","cited_by_count":160},{"id":"2109.14545","title":"Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark","created":"2021-09-29","cited_by_count":155},{"id":"2301.08727","title":"Neural Architecture Search: Insights from 1000 Papers","created":"2023-01-20","cited_by_count":139},{"id":"2006.12433","title":"What shapes feature representations? Exploring datasets, architectures, and training","created":"2020-06-22","cited_by_count":126},{"id":"2307.03381","title":"Teaching Arithmetic to Small Transformers","created":"2023-07-07","cited_by_count":123},{"id":"2207.10551","title":"Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?","created":"2022-07-21","cited_by_count":103},{"id":"2208.07998","title":"What Artificial Neural Networks Can Tell Us About Human Language Acquisition","created":"2022-08-17","cited_by_count":99},{"id":"2410.21272","title":"Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics","created":"2024-10-28","cited_by_count":97},{"id":"2310.04564","title":"ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models","created":"2023-10-06","cited_by_count":94},{"id":"1710.10686","title":"Regularization for Deep Learning: A Taxonomy","created":"2017-10-29","cited_by_count":91},{"id":"2005.00817","title":"A survey on modern trainable activation functions","created":"2020-05-02","cited_by_count":89},{"id":"2402.09371","title":"Transformers Can Achieve Length Generalization But Not Robustly","created":"2024-02-14","cited_by_count":86},{"id":"2105.05612","title":"Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization","created":"2021-05-12","cited_by_count":83},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83},{"id":"2402.09963","title":"Why are Sensitive Functions Hard for Transformers?","created":"2024-02-15","cited_by_count":74},{"id":"2211.12316","title":"Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions","created":"2022-11-22","cited_by_count":70},{"id":"2304.05366","title":"The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning","created":"2023-04-11","cited_by_count":63},{"id":"2501.17047","title":"How Linguistics Learned to Stop Worrying and Love the Language Models","created":"2025-01-28","cited_by_count":51},{"id":"2006.15191","title":"Is SGD a Bayesian sampler? Well, almost","created":"2020-06-26","cited_by_count":50},{"id":"2408.10914","title":"To Code, or Not To Code? Exploring Impact of Code in Pre-training","created":"2024-08-20","cited_by_count":47},{"id":"2403.02241","title":"Neural Redshift: Random Networks are not Random Functions","created":"2024-03-04","cited_by_count":43},{"id":"2306.13575","title":"Scaling MLPs: A Tale of Inductive Bias","created":"2023-06-23","cited_by_count":37},{"id":"2304.13060","title":"Injecting structural hints: Using language models to study inductive biases in language learning","created":"2023-04-25","cited_by_count":35},{"id":"1906.09529","title":"Learning Activation Functions: A new paradigm for understanding Neural Networks","created":"2019-06-23","cited_by_count":34},{"id":"2512.17351","title":"Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers","created":"2025-12-19","cited_by_count":33},{"id":"2308.12553","title":"Don't blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropy","created":"2023-08-24","cited_by_count":30},{"id":"2209.02681","title":"How important are activation functions in regression and classification? A survey, performance comparison, and future directions","created":"2022-09-06","cited_by_count":29},{"id":"2104.10343","title":"Sensitivity as a Complexity Measure for Sequence Classification Tasks","created":"2021-04-21","cited_by_count":28},{"id":"2409.04556","title":"How Does Code Pretraining Affect Language Model Task Performance?","created":"2024-09-06","cited_by_count":28},{"id":"2207.02598","title":"Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning","created":"2022-07-06","cited_by_count":26},{"id":"2011.14439","title":"Scaling Down Deep Learning with MNIST-1D","created":"2020-11-29","cited_by_count":25},{"id":"1909.11522","title":"Neural networks are a priori biased towards Boolean functions with low entropy","created":"2019-09-25","cited_by_count":24},{"id":"2210.16222","title":"Improving Lipschitz-Constrained Neural Networks by Learning Activation Functions","created":"2022-10-28","cited_by_count":24},{"id":"2402.05427","title":"A Sampling Theory Perspective on Activations for Implicit Neural Representations","created":"2024-02-08","cited_by_count":24},{"id":"2408.07144","title":"Language Models as Models of Language","created":"2024-08-13","cited_by_count":24},{"id":"2410.19637","title":"A distributional simplicity bias in the learning dynamics of transformers","created":"2024-10-25","cited_by_count":24},{"id":"2305.19905","title":"How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases","created":"2023-05-31","cited_by_count":23},{"id":"1801.09403","title":"Learning Combinations of Activation Functions","created":"2018-01-29","cited_by_count":20},{"id":"2502.18795","title":"Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMs","created":"2025-02-26","cited_by_count":20},{"id":"2507.13338","title":"Training Transformers with Enforced Lipschitz Constants","created":"2025-07-17","cited_by_count":20},{"id":"1809.06848","title":"On the Learning Dynamics of Deep Neural Networks","created":"2018-09-18","cited_by_count":18},{"id":"2002.07224","title":"Evolutionary Optimization of Deep Learning Activation Functions","created":"2020-02-17","cited_by_count":18},{"id":"2410.04368","title":"Algorithmic Capabilities of Random Transformers","created":"2024-10-06","cited_by_count":17},{"id":"2507.12224","title":"Optimizers Qualitatively Alter Solutions And We Should Leverage This","created":"2025-07-16","cited_by_count":17},{"id":"2502.12120","title":"LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws","created":"2025-02-17","cited_by_count":16},{"id":"1806.10064","title":"Adaptive Blending Units: Trainable Activation Functions for Deep Neural Networks","created":"2018-06-26","cited_by_count":13},{"id":"2503.10065","title":"Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild","created":"2025-03-13","cited_by_count":13},{"id":"2506.09251","title":"Extrapolation by Association: Length Generalization Transfer in Transformers","created":"2025-06-10","cited_by_count":13},{"id":"2505.17190","title":"Tropical Attention: Neural Algorithmic Reasoning for Combinatorial Algorithms","created":"2025-05-22","cited_by_count":12},{"id":"2212.06641","title":"Simplicity Bias Leads to Amplified Performance Disparities","created":"2022-12-13","cited_by_count":11},{"id":"1902.03306","title":"A simple and efficient architecture for trainable activation functions","created":"2019-02-08","cited_by_count":10},{"id":"2501.03489","title":"Entropy-Guided Attention for Private LLMs","created":"2025-01-07","cited_by_count":9},{"id":"2505.22308","title":"Transformers Pretrained on Procedural Data Contain Modular Structures for Algorithmic Reasoning","created":"2025-05-28","cited_by_count":9},{"id":"2407.07564","title":"Trainable Highly-expressive Activation Functions","created":"2024-07-10","cited_by_count":6},{"id":"2407.08567","title":"Adaptive Parametric Activation: Unifying and Generalising Activation Functions Across Tasks","created":"2024-07-11","cited_by_count":6},{"id":"2502.07783","title":"Curvature Tuning: Provable Training-free Model Steering From a Single Parameter","created":"2025-02-11","cited_by_count":4},{"id":"2507.20453","title":"Your Attention Matters: to Improve Model Robustness to Noise and Spurious Correlations","created":"2025-07-28","cited_by_count":3},{"id":"2507.20164","title":"ASNN: Learning to Suggest Neural Architectures from Performance Distributions","created":"2025-07-27","cited_by_count":1}],"next_cursor":null}