{"total":52,"rows":[{"id":"2503.09799","title":"Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo","created":"2025-03-12","cited_by_count":27,"area":null},{"id":"2412.13148","title":"SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training","created":"2024-12-17","cited_by_count":20,"area":null},{"id":"2410.03249","title":"How Much Can We Forget about Data Contamination?","created":"2024-10-04","cited_by_count":14,"area":null},{"id":"2502.15814","title":"Slamming: Training a Speech Language Model on One GPU in a Day","created":"2025-02-19","cited_by_count":12,"area":null},{"id":"2508.01483","title":"Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler","created":"2025-08-02","cited_by_count":12,"area":null},{"id":"2603.14315","title":"Enhancing LLM Training via Spectral Clipping","created":"2026-03-15","cited_by_count":10,"area":null},{"id":"2501.12243","title":"FOCUS: First Order Concentrated Updating Scheme","created":"2025-01-21","cited_by_count":8,"area":null},{"id":"2501.12391","title":"Physics of Skill Learning","created":"2025-01-21","cited_by_count":8,"area":null},{"id":"2505.24275","title":"GradPower: Powering Gradients for Faster Language Model Pre-Training","created":"2025-05-30","cited_by_count":8,"area":null},{"id":"2510.15830","title":"SNOO: Step-K Nesterov Outer Optimizer - The Surprising Effectiveness of Nesterov Momentum Applied to Pseudo-Gradients","created":"2025-10-17","cited_by_count":8,"area":null},{"id":"2602.09314","title":"Clarifying Shampoo: Adapting Spectral Descent to Stochasticity and the Parameter Trajectory","created":"2026-02-10","cited_by_count":8,"area":null},{"id":"2602.22681","title":"Accelerating LLM Pre-Training through Flat-Direction Dynamics Enhancement","created":"2026-02-26","cited_by_count":8,"area":null},{"id":"2509.23383","title":"Train Once, Answer All: Many Pretraining Experiments for the Cost of One","created":"2025-09-27","cited_by_count":6,"area":null},{"id":"2602.13498","title":"TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers","created":"2026-02-13","cited_by_count":6,"area":null},{"id":"2506.05447","title":"Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning","created":"2025-06-05","cited_by_count":5,"area":null},{"id":"2512.17131","title":"Smoothing DiLoCo with Primal Averaging for Faster Training of LLMs","created":"2025-12-18","cited_by_count":5,"area":null},{"id":"2604.15297","title":"Benchmarking Optimizers for MLPs in Tabular Deep Learning","created":"2026-04-16","cited_by_count":5,"area":null},{"id":"2505.22549","title":"DES-LOC: Desynced Low Communication Adaptive Optimizers for Training Foundation Models","created":"2025-05-28","cited_by_count":4,"area":null},{"id":"2602.01642","title":"The Effect of Mini-Batch Noise on the Implicit Bias of Adam","created":"2026-02-02","cited_by_count":4,"area":null},{"id":"2508.00180","title":"EMA Without the Lag: Bias-Corrected Iterate Averaging Schemes","created":"2025-07-31","cited_by_count":3,"area":null},{"id":"2510.14009","title":"Noise-Adaptive Layerwise Learning Rates: Accelerating Geometry-Aware Optimization for Deep Neural Network Training","created":"2025-10-15","cited_by_count":3,"area":null},{"id":"2605.11172","title":"Optimistic Dual Averaging Unifies Modern Optimizers","created":"2026-05-11","cited_by_count":3,"area":null},{"id":"2509.25380","title":"Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs","created":"2025-09-29","cited_by_count":2,"area":null},{"id":"2510.05361","title":"MT-DAO: Multi-Timescale Distributed Adaptive Optimizers with Local Updates","created":"2025-10-06","cited_by_count":2,"area":null},{"id":"2606.19179","title":"Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods","created":"2026-06-17","cited_by_count":2,"area":null},{"id":"2609.11655","title":"Musec: MomentUm SpEctral Clipping for Stable Muon-type Training","created":"2026-09-10","cited_by_count":2,"area":null},{"id":"2510.04988","title":"Adaptive Memory Momentum via a Model-Based Framework for Deep Learning Optimization","created":"2025-10-06","cited_by_count":1,"area":null},{"id":"2602.04396","title":"LoRDO: Distributed Low-Rank Optimization with Infrequent Communication","created":"2026-02-04","cited_by_count":1,"area":null},{"id":"2605.29128","title":"Apertus LLM Family Expansion via Distillation and Quantization","created":"2026-05-27","cited_by_count":1,"area":null},{"id":"2606.25086","title":"Training for the Model You Return: Improving Optimization for Iterate-Averaged Language Models","created":"2026-06-23","cited_by_count":1,"area":null},{"id":"2607.20486","title":"OPTScientist: Multi-Agent Discovery of Typed Optimizer Programs for Transformer Pretraining","created":"2026-06-02","cited_by_count":1,"area":null},{"id":"2608.18415","title":"The Road Taken: The Role of Optimizers at the Edge of Stability","created":"2026-08-19","cited_by_count":1,"area":null},{"id":"2609.04577","title":"Optimizer Memory Schedules for Outscaling the Overtraining Axis","created":"2026-09-04","cited_by_count":1,"area":null},{"id":"2609.30957","title":"Towards Understanding Momentum Acceleration in River-Valley Loss Landscape","created":"2026-09-25","cited_by_count":1,"area":null},{"id":"2410.05871","title":"A second-order-like optimizer with adaptive gradient scaling for deep learning","created":"2024-10-08","cited_by_count":0,"area":null},{"id":"2512.21792","title":"AI for Mycetoma Diagnosis in Histopathological Images: The MICCAI 2024 Challenge","created":"2025-12-25","cited_by_count":0,"area":null},{"id":"2603.04795","title":"LAW & ORDER: Adaptive Spatial Weighting for Medical Diffusion and Segmentation","created":"2026-03-05","cited_by_count":0,"area":null},{"id":"2604.20549","title":"Toward Cross-Lingual Quality Classifiers for Multilingual Pretraining Data Selection","created":"2026-04-22","cited_by_count":0,"area":null},{"id":"2605.19008","title":"Learn-by-Wire Training Control Governance: Bounded Autonomous Training Under Stress for Stability and Efficiency","created":"2026-05-18","cited_by_count":0,"area":null},{"id":"2608.21024","title":"RODE: A Radial-Orthogonal Decoupled Engine for Optimization","created":"2026-08-21","cited_by_count":0,"area":null},{"id":"2608.28442","title":"Curvature-Conditioned Multiscale Momentum with Sphere Constraints for LLM Pretraining","created":"2026-08-28","cited_by_count":0,"area":null},{"id":"2608.28557","title":"Blog: Survey of Optimizers","created":"2026-08-28","cited_by_count":0,"area":null},{"id":"2609.14339","title":"Communication-Efficient LLM Adaptation over Decentralized GPU Meshes","created":"2026-09-13","cited_by_count":0,"area":null},{"id":"2609.32290","title":"A Journey to the Edge of Stability","created":"2026-09-26","cited_by_count":0,"area":null},{"id":"2609.35701","title":"MeqMuon: Matrix-Equilibrating Muon for LLM Pretraining","created":"2026-09-28","cited_by_count":0,"area":null},{"id":"2609.36600","title":"Second-Moment Stochastic Approximation Methods","created":"2026-09-29","cited_by_count":0,"area":null},{"id":"2610.03432","title":"OptiSelect: How does the Optimizer Shape Data Curriculum?","created":"2026-10-02","cited_by_count":0,"area":null},{"id":"2610.03940","title":"A Step Towards Forgetting: Optimiser History and the Loss of Answer Mass","created":"2026-10-02","cited_by_count":0,"area":null},{"id":"2610.11585","title":"Adapting English Quality Classifiers for Multilingual LLM Pretraining Data Selection","created":"2026-10-08","cited_by_count":0,"area":null},{"id":"2610.13036","title":"Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf","created":"2026-10-09","cited_by_count":0,"area":null},{"id":"2610.14277","title":"Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and Inference","created":"2026-10-09","cited_by_count":0,"area":null},{"id":"2610.20413","title":"When Does Momentum Help in Reinforcement Learning with Verifiable Rewards?","created":"2026-10-14","cited_by_count":0,"area":null}],"next_cursor":null}