{"list":"other","total":75,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1503.02531","title":"Distilling the Knowledge in a Neural Network","created":"2015-03-09","cited_by_count":10324},{"id":"2110.14168","title":"Training Verifiers to Solve Math Word Problems","created":"2021-10-27","cited_by_count":10109},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2103.03874","title":"Measuring Mathematical Problem Solving With the MATH Dataset","created":"2021-03-05","cited_by_count":5371},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1910.01108","title":"DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter","created":"2019-10-02","cited_by_count":4307},{"id":"1612.01474","title":"Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles","created":"2016-12-05","cited_by_count":3926},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"1911.11641","title":"PIQA: Reasoning about Physical Commonsense in Natural Language","created":"2019-11-26","cited_by_count":2883},{"id":"2408.03314","title":"Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters","created":"2024-08-06","cited_by_count":1986},{"id":"1509.06461","title":"Deep Reinforcement Learning with Double Q-learning","created":"2015-09-22","cited_by_count":1745},{"id":"2203.05482","title":"Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time","created":"2022-03-10","cited_by_count":1396},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"2407.21787","title":"Large Language Monkeys: Scaling Inference Compute with Repeated Sampling","created":"2024-07-31","cited_by_count":991},{"id":"2306.13649","title":"On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes","created":"2023-06-23","cited_by_count":939},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"1606.07947","title":"Sequence-Level Knowledge Distillation","created":"2016-06-25","cited_by_count":801},{"id":"1707.06209","title":"Crowdsourcing Multiple Choice Science Questions","created":"2017-07-19","cited_by_count":765},{"id":"1912.02292","title":"Deep Double Descent: Where Bigger Models and More Data Hurt","created":"2019-12-04","cited_by_count":672},{"id":"1905.13319","title":"MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms","created":"2019-05-30","cited_by_count":662},{"id":"1802.10026","title":"Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs","created":"2018-02-27","cited_by_count":588},{"id":"1712.00409","title":"Deep Learning Scaling is Predictable, Empirically","created":"2017-12-01","cited_by_count":582},{"id":"2206.14486","title":"Beyond neural scaling laws: beating power law scaling via data pruning","created":"2022-06-29","cited_by_count":528},{"id":"2010.14701","title":"Scaling Laws for Autoregressive Generative Modeling","created":"2020-10-28","cited_by_count":522},{"id":"1903.08560","title":"Surprises in High-Dimensional Ridgeless Least Squares Interpolation","created":"2019-03-19","cited_by_count":513},{"id":"1905.08094","title":"Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation","created":"2019-05-17","cited_by_count":497},{"id":"1704.00109","title":"Snapshot Ensembles: Train 1, get M for free","created":"2017-04-01","cited_by_count":350},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"1812.06162","title":"An Empirical Model of Large-Batch Training","created":"2018-12-14","cited_by_count":281},{"id":"2309.14316","title":"Physics of Language Models: Part 3.1, Knowledge Storage and Extraction","created":"2023-09-25","cited_by_count":270},{"id":"1910.05653","title":"Model Fusion via Optimal Transport","created":"2019-10-12","cited_by_count":219},{"id":"1909.12673","title":"A Constructive Prediction of the Generalization Error Across Scales","created":"2019-09-27","cited_by_count":210},{"id":"1708.02182","title":"Regularizing and Optimizing LSTM Language Models","created":"2017-08-07","cited_by_count":205},{"id":"2211.04325","title":"Will we run out of data? Limits of LLM scaling based on human-generated data","created":"2022-10-26","cited_by_count":195},{"id":"1409.2329","title":"Recurrent Neural Network Regularization","created":"2014-09-08","cited_by_count":189},{"id":"2002.05715","title":"Self-Distillation Amplifies Regularization in Hilbert Space","created":"2020-02-13","cited_by_count":173},{"id":"2006.13198","title":"Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks","created":"2020-06-23","cited_by_count":166},{"id":"1512.05287","title":"A Theoretically Grounded Application of Dropout in Recurrent Neural Networks","created":"2015-12-16","cited_by_count":161},{"id":"2401.16380","title":"Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling","created":"2024-01-29","cited_by_count":161},{"id":"2412.02595","title":"Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset","created":"2024-12-03","cited_by_count":160},{"id":"2401.00448","title":"Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws","created":"2023-12-31","cited_by_count":159},{"id":"1711.03953","title":"Breaking the Softmax Bottleneck: A High-Rank RNN Language Model","created":"2017-11-10","cited_by_count":152},{"id":"2404.01413","title":"Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data","created":"2024-04-01","cited_by_count":151},{"id":"2405.10938","title":"Observational Scaling Laws and the Predictability of Language Model Performance","created":"2024-05-17","cited_by_count":130},{"id":"2003.01897","title":"Optimal Regularization Can Mitigate Double Descent","created":"2020-03-04","cited_by_count":109},{"id":"2509.02046","title":"Fantastic Pretraining Optimizers and Where to Find Them","created":"2025-09-02","cited_by_count":100},{"id":"1612.04426","title":"Improving Neural Language Models with a Continuous Cache","created":"2016-12-13","cited_by_count":82},{"id":"2006.15081","title":"On the Generalization Benefit of Noise in Stochastic Gradient Descent","created":"2020-06-26","cited_by_count":82},{"id":"1703.02573","title":"Data Noising as Smoothing in Neural Network Language Models","created":"2017-03-07","cited_by_count":78},{"id":"2404.07177","title":"Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic","created":"2024-04-10","cited_by_count":77},{"id":"2407.05872","title":"Scaling Exponents Across Parameterizations and Optimizers","created":"2024-07-08","cited_by_count":76},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72},{"id":"2507.15857","title":"Diffusion Beats Autoregressive in Data-Constrained Settings","created":"2025-07-21","cited_by_count":63},{"id":"2301.11796","title":"Call for Papers -- The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus","created":"2023-01-27","cited_by_count":62},{"id":"2209.03942","title":"Data Feedback Loops: Model-driven Amplification of Dataset Biases","created":"2022-09-08","cited_by_count":60},{"id":"2503.19206","title":"Overtrained Language Models Are Harder to Fine-Tune","created":"2025-03-24","cited_by_count":58},{"id":"1709.07432","title":"Dynamic Evaluation of Neural Sequence Models","created":"2017-09-21","cited_by_count":55},{"id":"2103.04514","title":"Nondeterminism and Instability in Neural Network Optimization","created":"2021-03-08","cited_by_count":48},{"id":"2410.08527","title":"Scaling Laws for Predicting Downstream Performance in LLMs","created":"2024-10-11","cited_by_count":48},{"id":"2409.05816","title":"Improving Pretraining Data Using Perplexity Correlations","created":"2024-09-09","cited_by_count":46},{"id":"2503.18866","title":"Reasoning to Learn from Latent Thoughts","created":"2025-03-24","cited_by_count":45},{"id":"2305.18411","title":"Feature-Learning Networks Are Consistent Across Widths At Realistic Scales","created":"2023-05-28","cited_by_count":41},{"id":"2507.07101","title":"Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful","created":"2025-07-09","cited_by_count":41},{"id":"1607.03474","title":"Recurrent Highway Networks","created":"2016-07-12","cited_by_count":37},{"id":"2507.02092","title":"Energy-Based Transformers are Scalable Learners and Thinkers","created":"2025-07-02","cited_by_count":31},{"id":"2007.08483","title":"On Power Laws in Deep Ensembles","created":"2020-07-16","cited_by_count":26},{"id":"2101.00411","title":"Substructure Substitution: Structured Data Augmentation for NLP","created":"2021-01-02","cited_by_count":25},{"id":"2304.01910","title":"On the Variance of Neural Network Training with respect to Test Sets and Distributions","created":"2023-04-04","cited_by_count":22},{"id":"1909.01736","title":"Beyond Human-Level Accuracy: Computational Challenges in Deep Learning","created":"2019-09-03","cited_by_count":19},{"id":"2311.14646","title":"More is Better in Modern Machine Learning: when Infinite Overparameterization is Optimal and Overfitting is Obligatory","created":"2023-11-24","cited_by_count":18},{"id":"1601.04650","title":"Statistical Mechanics of High-Dimensional Inference","created":"2016-01-18","cited_by_count":13},{"id":"1808.10143","title":"Direct Output Connection for a High-Rank Language Model","created":"2018-08-30","cited_by_count":11},{"id":"2509.01649","title":"Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling","created":"2025-09-01","cited_by_count":7},{"id":"2412.05418","title":"No Free Lunch From Random Feature Ensembles: Scaling Laws and Near-Optimality Conditions","created":"2024-12-06","cited_by_count":1}],"next_cursor":null}