{"list":"other","total":58,"rows":[{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"2108.07732","title":"Program Synthesis with Large Language Models","created":"2021-08-16","cited_by_count":3637},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"2304.01373","title":"Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling","created":"2023-04-03","cited_by_count":1786},{"id":"2012.14913","title":"Transformer Feed-Forward Layers Are Key-Value Memories","created":"2020-12-29","cited_by_count":1336},{"id":"2102.04664","title":"CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation","created":"2021-02-09","cited_by_count":744},{"id":"2304.14997","title":"Towards Automated Circuit Discovery for Mechanistic Interpretability","created":"2023-04-28","cited_by_count":724},{"id":"1912.00582","title":"BLiMP: The Benchmark of Linguistic Minimal Pairs for English","created":"2019-12-02","cited_by_count":564},{"id":"2305.10429","title":"DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining","created":"2023-05-17","cited_by_count":399},{"id":"2405.07987","title":"The Platonic Representation Hypothesis","created":"2024-05-13","cited_by_count":382},{"id":"2112.10510","title":"Transformers Can Do Bayesian Inference","created":"2021-12-20","cited_by_count":360},{"id":"1904.01557","title":"Analysing Mathematical Reasoning Abilities of Neural Models","created":"2019-04-02","cited_by_count":335},{"id":"2205.05055","title":"Data Distributional Properties Drive Emergent In-Context Learning in Transformers","created":"2022-04-22","cited_by_count":309},{"id":"2305.13169","title":"A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity","created":"2023-05-22","cited_by_count":182},{"id":"2410.21272","title":"Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics","created":"2024-10-28","cited_by_count":97},{"id":"2106.05963","title":"Learning to See by Looking at Noise","created":"2021-06-10","cited_by_count":92},{"id":"2310.15393","title":"DoGE: Domain Reweighting with Generalization Estimation","created":"2023-10-23","cited_by_count":90},{"id":"2306.00946","title":"Exposing Attention Glitches with Flip-Flop Language Modeling","created":"2023-06-01","cited_by_count":75},{"id":"2305.09828","title":"Mimetic Initialization of Self-Attention Layers","created":"2023-05-16","cited_by_count":54},{"id":"2408.10914","title":"To Code, or Not To Code? Exploring Impact of Code in Pre-training","created":"2024-08-20","cited_by_count":47},{"id":"2101.06223","title":"LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning","created":"2021-01-15","cited_by_count":46},{"id":"2403.02241","title":"Neural Redshift: Random Networks are not Random Functions","created":"2024-03-04","cited_by_count":43},{"id":"2305.14701","title":"Modeling rapid language learning by distilling Bayesian priors into artificial neural networks","created":"2023-05-24","cited_by_count":42},{"id":"2006.00555","title":"Transferring Inductive Biases through Knowledge Distillation","created":"2020-05-31","cited_by_count":41},{"id":"2411.12580","title":"Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models","created":"2024-11-19","cited_by_count":41},{"id":"2311.18823","title":"Initializing Models with Larger Ones","created":"2023-11-30","cited_by_count":39},{"id":"2502.19249","title":"Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases","created":"2025-02-26","cited_by_count":37},{"id":"2304.13060","title":"Injecting structural hints: Using language models to study inductive biases in language learning","created":"2023-04-25","cited_by_count":35},{"id":"2401.14953","title":"Learning Universal Predictors","created":"2024-01-26","cited_by_count":31},{"id":"2203.10326","title":"Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models","created":"2022-03-19","cited_by_count":28},{"id":"2211.16412","title":"Procedural Image Programs for Representation Learning","created":"2022-11-29","cited_by_count":28},{"id":"2409.04556","title":"How Does Code Pretraining Affect Language Model Task Performance?","created":"2024-09-06","cited_by_count":28},{"id":"2206.10139","title":"Insights into Pre-training via Simpler Synthetic Tasks","created":"2022-06-21","cited_by_count":26},{"id":"2109.03537","title":"On the Transferability of Pre-trained Language Models: A Study from Artificial Datasets","created":"2021-09-08","cited_by_count":25},{"id":"2505.11581","title":"Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis","created":"2025-05-16","cited_by_count":22},{"id":"2209.14389","title":"Downstream Datasets Make Surprisingly Good Pretraining Corpora","created":"2022-09-28","cited_by_count":20},{"id":"2408.00677","title":"Scaling Backwards: Minimal Synthetic Pre-training?","created":"2024-08-01","cited_by_count":14},{"id":"2505.24844","title":"Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning","created":"2025-05-30","cited_by_count":14},{"id":"2510.02375","title":"Pretraining with hierarchical memories: separating long-tail and common knowledge","created":"2025-09-29","cited_by_count":14},{"id":"2410.02536","title":"Intelligence at the Edge of Chaos","created":"2024-10-03","cited_by_count":13},{"id":"2503.10065","title":"Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild","created":"2025-03-13","cited_by_count":13},{"id":"2502.19402","title":"General Intelligence Requires Reward-based Pretraining","created":"2025-02-26","cited_by_count":12},{"id":"2506.01115","title":"Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer","created":"2025-06-01","cited_by_count":12},{"id":"2410.07041","title":"Emergent properties with repeated examples","created":"2024-10-09","cited_by_count":11},{"id":"2310.00771","title":"Pre-training with Synthetic Data Helps Offline Reinforcement Learning","created":"2023-10-01","cited_by_count":9},{"id":"2207.00052","title":"Visual Pre-training for Navigation: What Can We Learn from Noise?","created":"2022-06-30","cited_by_count":8},{"id":"2212.09864","title":"Synthetic Pre-Training Tasks for Neural Machine Translation","created":"2022-12-19","cited_by_count":8},{"id":"2503.16048","title":"Meta-Learning Neural Mechanisms rather than Bayesian Priors","created":"2025-03-20","cited_by_count":8},{"id":"2511.13945","title":"Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers","created":"2025-11-17","cited_by_count":8},{"id":"2310.00796","title":"SIP: Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulation","created":"2023-10-01","cited_by_count":7},{"id":"2310.10899","title":"Instilling Inductive Biases with Subnetworks","created":"2023-10-17","cited_by_count":6},{"id":"2506.20057","title":"Universal pre-training by iterated random computation","created":"2025-06-24","cited_by_count":5},{"id":"2012.02550","title":"Effect of the initial configuration of weights on the training and function of artificial neural networks","created":"2020-12-04","cited_by_count":4}],"next_cursor":null}