{"list":"other","total":63,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1909.08053","title":"Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism","created":"2019-09-17","cited_by_count":1868},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"2106.04560","title":"Scaling Vision Transformers","created":"2021-06-08","cited_by_count":999},{"id":"2404.06395","title":"MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies","created":"2024-04-09","cited_by_count":723},{"id":"2401.02954","title":"DeepSeek LLM: Scaling Open-Source Language Models with Longtermism","created":"2024-01-05","cited_by_count":642},{"id":"2104.04473","title":"Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM","created":"2021-04-09","cited_by_count":632},{"id":"1712.00409","title":"Deep Learning Scaling is Predictable, Empirically","created":"2017-12-01","cited_by_count":582},{"id":"2010.14701","title":"Scaling Laws for Autoregressive Generative Modeling","created":"2020-10-28","cited_by_count":522},{"id":"2508.06471","title":"GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models","created":"2025-08-08","cited_by_count":467},{"id":"2403.03507","title":"GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection","created":"2024-03-06","cited_by_count":463},{"id":"2305.16264","title":"Scaling Data-Constrained Language Models","created":"2023-05-25","cited_by_count":403},{"id":"2102.06701","title":"Explaining Neural Scaling Laws","created":"2021-02-12","cited_by_count":363},{"id":"1711.00489","title":"Don't Decay the Learning Rate, Increase the Batch Size","created":"2017-11-01","cited_by_count":320},{"id":"1812.06162","title":"An Empirical Model of Large-Batch Training","created":"2018-12-14","cited_by_count":281},{"id":"2405.18392","title":"Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations","created":"2024-05-28","cited_by_count":170},{"id":"2501.08313","title":"MiniMax-01: Scaling Foundation Models with Lightning Attention","created":"2025-01-14","cited_by_count":166},{"id":"2401.00448","title":"Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws","created":"2023-12-31","cited_by_count":159},{"id":"1712.06559","title":"The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning","created":"2017-12-18","cited_by_count":150},{"id":"2406.11704","title":"Nemotron-4 340B Technical Report","created":"2024-06-17","cited_by_count":144},{"id":"2301.03728","title":"Scaling Laws for Generative Mixed-Modal Language Models","created":"2023-01-10","cited_by_count":134},{"id":"1811.01558","title":"Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations","created":"2018-11-05","cited_by_count":114},{"id":"2402.01092","title":"A Dynamical Model of Neural Scaling Laws","created":"2024-02-02","cited_by_count":111},{"id":"2410.05192","title":"Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective","created":"2024-10-07","cited_by_count":81},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73},{"id":"1805.10074","title":"Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes","created":"2018-05-25","cited_by_count":69},{"id":"2405.15074","title":"4+3 Phases of Compute-Optimal Neural Scaling Laws","created":"2024-05-23","cited_by_count":68},{"id":"2207.02628","title":"The alignment property of SGD noise and how it helps select flat minima: A stability analysis","created":"2022-07-06","cited_by_count":64},{"id":"2509.01322","title":"LongCat-Flash Technical Report","created":"2025-09-01","cited_by_count":58},{"id":"2406.08466","title":"Scaling Laws in Linear Regression: Compute, Parameters, and Data","created":"2024-06-12","cited_by_count":57},{"id":"2409.17858","title":"How Feature Learning Can Improve Neural Scaling Laws","created":"2024-09-26","cited_by_count":56},{"id":"2412.05270","title":"APOLLO: SGD-like Memory, AdamW-level Performance","created":"2024-12-06","cited_by_count":54},{"id":"2310.07831","title":"Optimal Linear Decay Learning Rate Schedules and Further Refinements","created":"2023-10-11","cited_by_count":43},{"id":"2110.06198","title":"Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression","created":"2021-10-12","cited_by_count":38},{"id":"2402.16819","title":"Nemotron-4 15B Technical Report","created":"2024-02-26","cited_by_count":36},{"id":"2404.05019","title":"Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts","created":"2024-04-07","cited_by_count":32},{"id":"2502.19002","title":"The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training","created":"2025-02-26","cited_by_count":30},{"id":"2105.09557","title":"Power-law escape rate of SGD","created":"2021-05-20","cited_by_count":29},{"id":"2408.11029","title":"Scaling Law with Learning Rate Annealing","created":"2024-08-20","cited_by_count":29},{"id":"2505.19227","title":"Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law","created":"2025-05-25","cited_by_count":24},{"id":"2505.23971","title":"Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training","created":"2025-05-29","cited_by_count":23},{"id":"2503.12811","title":"A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules","created":"2025-03-17","cited_by_count":22},{"id":"2507.02119","title":"Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks","created":"2025-07-02","cited_by_count":22},{"id":"1902.08668","title":"Beating SGD Saturation with Tail-Averaging and Minibatching","created":"2019-02-22","cited_by_count":21},{"id":"2206.01029","title":"Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions","created":"2022-06-02","cited_by_count":19},{"id":"1810.02565","title":"Continuous-time Models for Stochastic Optimization Algorithms","created":"2018-10-05","cited_by_count":17},{"id":"2509.19189","title":"Functional Scaling Laws in Kernel Regression: Loss Dynamics and Learning Rate Schedules","created":"2025-09-23","cited_by_count":17},{"id":"2511.13421","title":"Larger Datasets Can Be Repeated More: A Theoretical Analysis of Multi-Epoch Scaling in Linear Regression","created":"2025-11-17","cited_by_count":16},{"id":"2602.05725","title":"Muon in Associative Memory Learning: Training Dynamics and Scaling Laws","created":"2026-02-05","cited_by_count":13},{"id":"2310.00692","title":"A Theoretical Analysis of Noise Geometry in Stochastic Gradient Descent","created":"2023-10-01","cited_by_count":9},{"id":"2402.11215","title":"AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods","created":"2024-02-17","cited_by_count":8},{"id":"2505.24275","title":"GradPower: Powering Gradients for Faster Language Model Pre-Training","created":"2025-05-30","cited_by_count":8},{"id":"2602.06797","title":"Optimal Learning Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-Decay","created":"2026-02-06","cited_by_count":8},{"id":"2411.00999","title":"Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers","created":"2024-11-01","cited_by_count":7},{"id":"2412.21124","title":"Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism","created":"2024-12-30","cited_by_count":4},{"id":"2312.01898","title":"Unlocking optimal batch size schedules using continuous-time control and perturbation theory","created":"2023-12-04","cited_by_count":3}],"next_cursor":null}