{"list":"other","total":66,"rows":[{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"1608.03983","title":"SGDR: Stochastic Gradient Descent with Warm Restarts","created":"2016-08-13","cited_by_count":5045},{"id":"1612.00796","title":"Overcoming catastrophic forgetting in neural networks","created":"2016-12-02","cited_by_count":4787},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"2307.15818","title":"RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control","created":"2023-07-28","cited_by_count":3080},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"2406.09246","title":"OpenVLA: An Open-Source Vision-Language-Action Model","created":"2024-06-13","cited_by_count":2887},{"id":"2303.18223","title":"A Survey of Large Language Models","created":"2023-03-31","cited_by_count":2576},{"id":"2006.16668","title":"GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding","created":"2020-06-30","cited_by_count":1631},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"1910.07467","title":"Root Mean Square Layer Normalization","created":"2019-10-16","cited_by_count":1480},{"id":"1802.07569","title":"Continual Lifelong Learning with Neural Networks: A Review","created":"2018-02-21","cited_by_count":1330},{"id":"1803.05407","title":"Averaging Weights Leads to Wider Optima and Better Generalization","created":"2018-03-14","cited_by_count":1209},{"id":"2406.17557","title":"The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale","created":"2024-06-25","cited_by_count":1160},{"id":"2302.00487","title":"A Comprehensive Survey of Continual Learning: Theory, Method and Application","created":"2023-01-31","cited_by_count":872},{"id":"2404.06395","title":"MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies","created":"2024-04-09","cited_by_count":723},{"id":"2402.06196","title":"Large Language Models: A Survey","created":"2024-02-09","cited_by_count":525},{"id":"2602.02276","title":"Kimi K2.5: Visual Agentic Intelligence","created":"2026-02-02","cited_by_count":501},{"id":"2302.06675","title":"Symbolic Discovery of Optimization Algorithms","created":"2023-02-13","cited_by_count":434},{"id":"2502.16982","title":"Muon is Scalable for LLM Training","created":"2025-02-24","cited_by_count":428},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"1901.10159","title":"An Investigation into Neural Net Optimization via Hessian Eigenvalue Density","created":"2019-01-29","cited_by_count":292},{"id":"1706.04454","title":"Empirical Analysis of the Hessian of Over-Parametrized Neural Networks","created":"2017-06-14","cited_by_count":290},{"id":"2405.14093","title":"A Survey on Vision-Language-Action Models for Embodied AI","created":"2024-05-23","cited_by_count":279},{"id":"2409.20325","title":"Old Optimizer, New Norm: An Anthology","created":"2024-09-30","cited_by_count":193},{"id":"2409.11321","title":"SOAP: Improving and Stabilizing Shampoo using Adam","created":"2024-09-17","cited_by_count":192},{"id":"2502.07529","title":"Training Deep Learning Models with Norm-Constrained LMOs","created":"2025-02-11","cited_by_count":192},{"id":"2405.18392","title":"Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations","created":"2024-05-28","cited_by_count":170},{"id":"2405.15682","title":"The Road Less Scheduled","created":"2024-05-24","cited_by_count":163},{"id":"1706.05350","title":"L2 Regularization versus Batch and Weight Normalization","created":"2017-06-16","cited_by_count":124},{"id":"2301.07733","title":"Learning-Rate-Free Learning by D-Adaptation","created":"2023-01-18","cited_by_count":113},{"id":"2505.16932","title":"The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm","created":"2025-05-22","cited_by_count":110},{"id":"2510.05491","title":"NorMuon: Making Muon more efficient and scalable","created":"2025-10-07","cited_by_count":84},{"id":"2502.02900","title":"A Note on the Convergence of Muon","created":"2025-02-05","cited_by_count":83},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73},{"id":"2504.05295","title":"Dion: Distributed Orthonormalized Updates","created":"2025-04-07","cited_by_count":68},{"id":"2509.01440","title":"Benchmarking Optimizers for Large Language Model Pretraining","created":"2025-09-01","cited_by_count":68},{"id":"2402.03804","title":"ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs","created":"2024-02-06","cited_by_count":67},{"id":"2507.11005","title":"AdaMuon: Adaptive Muon Optimizer","created":"2025-07-15","cited_by_count":67},{"id":"2503.20762","title":"ASGO: Adaptive Structured Gradient Optimization","created":"2025-03-26","cited_by_count":57},{"id":"2406.09405","title":"Why Warmup the Learning Rate? Underlying Mechanisms and Improvements","created":"2024-06-13","cited_by_count":46},{"id":"2505.12082","title":"Model Merging in Pre-training of Large Language Models","created":"2025-05-17","cited_by_count":37},{"id":"2510.16981","title":"MuonBP: Faster Muon via Block-Periodic Orthogonalization","created":"2025-10-19","cited_by_count":35},{"id":"2501.18965","title":"The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training","created":"2025-01-31","cited_by_count":33},{"id":"2502.15015","title":"Accelerating Neural Network Training: An Analysis of the AlgoPerf Competition","created":"2025-02-20","cited_by_count":32},{"id":"2509.15816","title":"On the Convergence of Muon and Beyond","created":"2025-09-19","cited_by_count":31},{"id":"2512.04299","title":"When do spectral gradient updates help in deep learning?","created":"2025-12-03","cited_by_count":31},{"id":"2507.01598","title":"Convergence Bound and Critical Batch Size of Muon Optimizer","created":"2025-07-02","cited_by_count":28},{"id":"2506.02285","title":"Why Gradients Rapidly Increase Near the End of Training","created":"2025-06-02","cited_by_count":22},{"id":"2512.05620","title":"Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales","created":"2025-12-05","cited_by_count":21},{"id":"2604.09967","title":"Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning","created":"2026-04-11","cited_by_count":17},{"id":"2512.16928","title":"Dion2: A Simple Method to Shrink Matrix in Muon","created":"2025-12-01","cited_by_count":13},{"id":"2508.01483","title":"Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler","created":"2025-08-02","cited_by_count":12},{"id":"2603.28743","title":"Rethinking Language Model Scaling under Transferable Hypersphere Optimization","created":"2026-03-30","cited_by_count":12},{"id":"2602.17004","title":"Arcee Trinity Large Technical Report","created":"2026-02-19","cited_by_count":11},{"id":"2602.03702","title":"Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging","created":"2026-02-03","cited_by_count":10},{"id":"2507.09846","title":"Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training","created":"2025-07-14","cited_by_count":9},{"id":"2401.17401","title":"Step-size Optimization for Continual Learning","created":"2024-01-30","cited_by_count":6},{"id":"2512.09678","title":"Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization","created":"2025-12-10","cited_by_count":4},{"id":"2505.24005","title":"How far away are truly hyperparameter-free learning algorithms?","created":"2025-05-29","cited_by_count":2}],"next_cursor":null}