{"list":"other","total":51,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2507.06261","title":"Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities","created":"2025-07-07","cited_by_count":3796},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"2304.01373","title":"Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling","created":"2023-04-03","cited_by_count":1786},{"id":"2111.02114","title":"LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs","created":"2021-11-03","cited_by_count":1555},{"id":"2112.11446","title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","created":"2021-12-08","cited_by_count":1202},{"id":"2305.10403","title":"PaLM 2 Technical Report","created":"2023-05-17","cited_by_count":1162},{"id":"2306.01116","title":"The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only","created":"2023-06-01","cited_by_count":763},{"id":"2404.06395","title":"MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies","created":"2024-04-09","cited_by_count":723},{"id":"2412.08905","title":"Phi-4 Technical Report","created":"2024-12-12","cited_by_count":711},{"id":"1904.11955","title":"On Exact Computation with an Infinitely Wide Neural Net","created":"2019-04-26","cited_by_count":573},{"id":"1911.00359","title":"CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data","created":"2019-11-01","cited_by_count":548},{"id":"2311.16867","title":"The Falcon Series of Open Language Models","created":"2023-11-28","cited_by_count":535},{"id":"2305.10429","title":"DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining","created":"2023-05-17","cited_by_count":399},{"id":"2406.11794","title":"DataComp-LM: In search of the next generation of training sets for language models","created":"2024-06-17","cited_by_count":398},{"id":"1906.11829","title":"Selection via Proxy: Efficient Data Selection for Deep Learning","created":"2019-06-26","cited_by_count":339},{"id":"1711.00489","title":"Don't Decay the Learning Rate, Increase the Batch Size","created":"2017-11-01","cited_by_count":320},{"id":"1911.02590","title":"Optimizing Millions of Hyperparameters by Implicit Differentiation","created":"2019-11-06","cited_by_count":304},{"id":"1704.04289","title":"Stochastic Gradient Descent as Approximate Bayesian Inference","created":"2017-04-13","cited_by_count":300},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"1812.06162","title":"An Empirical Model of Large-Batch Training","created":"2018-12-14","cited_by_count":281},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"2103.12828","title":"Learning to Optimize: A Primer and A Benchmark","created":"2021-03-23","cited_by_count":208},{"id":"2403.16952","title":"Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance","created":"2024-03-25","cited_by_count":184},{"id":"2411.12372","title":"RedPajama: an Open Dataset for Training Large Language Models","created":"2024-11-19","cited_by_count":182},{"id":"2407.01492","title":"RegMix: Data Mixture as Regression for Language Model Pre-training","created":"2024-07-01","cited_by_count":177},{"id":"2307.14430","title":"Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models","created":"2023-07-26","cited_by_count":121},{"id":"2402.01092","title":"A Dynamical Model of Neural Scaling Laws","created":"2024-02-02","cited_by_count":111},{"id":"2207.10062","title":"DataPerf: Benchmarks for Data-Centric AI Development","created":"2022-07-20","cited_by_count":90},{"id":"2310.15393","title":"DoGE: Domain Reweighting with Generalization Estimation","created":"2023-10-23","cited_by_count":90},{"id":"2401.08565","title":"Tuning Language Models by Proxy","created":"2024-01-16","cited_by_count":89},{"id":"2002.11770","title":"Rethinking the Hyperparameters for Fine-tuning","created":"2020-02-19","cited_by_count":69},{"id":"2312.03724","title":"DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer","created":"2023-11-27","cited_by_count":57},{"id":"2406.08466","title":"Scaling Laws in Linear Regression: Compute, Parameters, and Data","created":"2024-06-12","cited_by_count":57},{"id":"1904.06963","title":"The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent","created":"2019-04-15","cited_by_count":46},{"id":"2305.18411","title":"Feature-Learning Networks Are Consistent Across Widths At Realistic Scales","created":"2023-05-28","cited_by_count":41},{"id":"2504.11393","title":"DataDecide: How to Predict Best Pretraining Data with Small Experiments","created":"2025-04-15","cited_by_count":40},{"id":"2402.10430","title":"Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models","created":"2024-02-16","cited_by_count":32},{"id":"2406.09334","title":"ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models","created":"2024-06-13","cited_by_count":20},{"id":"2505.18091","title":"Data Mixing Can Induce Phase Transitions in Knowledge Acquisition","created":"2025-05-23","cited_by_count":16},{"id":"2410.18779","title":"A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs","created":"2024-10-24","cited_by_count":15},{"id":"2411.12925","title":"Loss-to-Loss Prediction: Scaling Laws for All Datasets","created":"2024-11-19","cited_by_count":15},{"id":"2503.02877","title":"Weak-to-Strong Generalization Even in Random Feature Networks, Provably","created":"2025-03-04","cited_by_count":15},{"id":"2406.04606","title":"Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions","created":"2024-06-07","cited_by_count":14},{"id":"2410.02498","title":"Dynamic Gradient Alignment for Online Data Mixing","created":"2024-10-03","cited_by_count":10},{"id":"2403.01101","title":"Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models","created":"2024-03-02","cited_by_count":4},{"id":"2412.09990","title":"Small Language Model as Data Prospector for Large Language Model","created":"2024-12-13","cited_by_count":2}],"next_cursor":null}