{"total":62,"rows":[{"id":"2511.00674","title":"Isotropic Curvature Model for Understanding Deep Learning Optimization: Is Gradient Orthogonalization Optimal?","created":"2025-11-01","cited_by_count":37,"area":null},{"id":"2512.05620","title":"Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales","created":"2025-12-05","cited_by_count":21,"area":null},{"id":"2510.22980","title":"How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data","created":"2025-10-27","cited_by_count":19,"area":null},{"id":"2510.25000","title":"What Really Matters in Matrix-Whitening Optimizers?","created":"2025-10-28","cited_by_count":16,"area":null},{"id":"2603.26554","title":"Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory","created":"2026-03-27","cited_by_count":14,"area":null},{"id":"2605.11181","title":"Muon is Not That Special: Random or Inverted Spectra Work Just as Well","created":"2026-05-11","cited_by_count":14,"area":null},{"id":"2601.23000","title":"Mano: Restriking Manifold Optimization for LLM Training","created":"2026-01-30","cited_by_count":12,"area":null},{"id":"2603.10067","title":"HTMuon: Improving Muon via Heavy-Tailed Spectral Correction","created":"2026-03-10","cited_by_count":12,"area":null},{"id":"2510.10777","title":"Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods","created":"2025-10-12","cited_by_count":11,"area":null},{"id":"2510.21800","title":"MARS-M: When Variance Reduction Meets Matrices","created":"2025-10-20","cited_by_count":10,"area":null},{"id":"2603.14315","title":"Enhancing LLM Training via Spectral Clipping","created":"2026-03-15","cited_by_count":10,"area":null},{"id":"2509.26337","title":"FedMuon: Federated Learning with Bias-corrected LMO-based Optimization","created":"2025-09-30","cited_by_count":8,"area":null},{"id":"2510.03871","title":"Optimal Scaling Needs Optimal Norm","created":"2025-10-04","cited_by_count":8,"area":null},{"id":"2602.06797","title":"Optimal Learning Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-Decay","created":"2026-02-06","cited_by_count":8,"area":null},{"id":"2602.09314","title":"Clarifying Shampoo: Adapting Spectral Descent to Stochasticity and the Parameter Trajectory","created":"2026-02-10","cited_by_count":8,"area":null},{"id":"2509.23500","title":"Beyond Outliers: A Study of Optimizers Under Quantization","created":"2025-09-27","cited_by_count":6,"area":null},{"id":"2602.07712","title":"Towards Robust Scaling Laws for Optimizers","created":"2026-02-07","cited_by_count":6,"area":null},{"id":"2602.05813","title":"Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers","created":"2026-02-05","cited_by_count":5,"area":null},{"id":"2604.15297","title":"Benchmarking Optimizers for MLPs in Tabular Deep Learning","created":"2026-04-16","cited_by_count":5,"area":null},{"id":"2604.15416","title":"StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models","created":"2026-04-16","cited_by_count":5,"area":null},{"id":"2605.06654","title":"Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less","created":"2026-05-07","cited_by_count":5,"area":null},{"id":"2603.03597","title":"NuMuon: Nuclear-Norm-Constrained Muon for Compressible LLM Training","created":"2026-03-04","cited_by_count":4,"area":null},{"id":"2604.09258","title":"Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima","created":"2026-04-10","cited_by_count":4,"area":null},{"id":"2605.22432","title":"AMUSE: Anytime Muon with Stable Gradient Evaluation","created":"2026-05-21","cited_by_count":4,"area":null},{"id":"2605.31371","title":"Softsign: Smooth Sign in Your Optimizer For Better Parameter Heterogeneity Handling","created":"2026-05-29","cited_by_count":4,"area":null},{"id":"2604.12968","title":"Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations","created":"2026-04-14","cited_by_count":3,"area":null},{"id":"2605.17109","title":"DynMuon: A Dynamic Spectral Shaping View of Muon","created":"2026-05-16","cited_by_count":3,"area":null},{"id":"2605.23061","title":"Anytime Training with Schedule-Free Spectral Optimization","created":"2026-05-21","cited_by_count":3,"area":null},{"id":"2510.05361","title":"MT-DAO: Multi-Timescale Distributed Adaptive Optimizers with Local Updates","created":"2025-10-06","cited_by_count":2,"area":null},{"id":"2605.06615","title":"When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\\ell_1$-norm Lower Bounds","created":"2026-05-07","cited_by_count":2,"area":null},{"id":"2605.09176","title":"Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers","created":"2026-05-09","cited_by_count":2,"area":null},{"id":"2605.17787","title":"Revisiting the Adam-SGD Gap in LLM Pre-Training: The Role of Large Effective Learning Rates","created":"2026-05-18","cited_by_count":2,"area":null},{"id":"2605.19811","title":"LionMuon: Alternating Spectral and Sign Descent for Efficient Training","created":"2026-05-19","cited_by_count":2,"area":null},{"id":"2606.03899","title":"Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering","created":"2026-06-02","cited_by_count":2,"area":null},{"id":"2606.15268","title":"When to use what Schatten-$p$ norm in deep learning?","created":"2026-06-13","cited_by_count":2,"area":null},{"id":"2606.27216","title":"Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization","created":"2026-06-25","cited_by_count":2,"area":null},{"id":"2607.17620","title":"PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer","created":"2026-07-20","cited_by_count":2,"area":null},{"id":"2607.23777","title":"Scale Weight Decay and Train Better","created":"2026-07-26","cited_by_count":2,"area":null},{"id":"2511.06477","title":"DyKAF: Dynamical Kronecker Approximation of the Fisher Information Matrix for Gradient Preconditioning","created":"2025-11-09","cited_by_count":1,"area":null},{"id":"2602.04396","title":"LoRDO: Distributed Low-Rank Optimization with Infrequent Communication","created":"2026-02-04","cited_by_count":1,"area":null},{"id":"2606.23676","title":"Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?","created":"2026-06-22","cited_by_count":1,"area":null},{"id":"2607.10611","title":"M+Adam: Low-Precision Training via Additive-Multiplicative Optimization","created":"2026-07-12","cited_by_count":1,"area":null},{"id":"2609.04577","title":"Optimizer Memory Schedules for Outscaling the Overtraining Axis","created":"2026-09-04","cited_by_count":1,"area":null},{"id":"2601.01306","title":"Towards a Principled Muon under $\\mu\\mathsf{P}$: Ensuring Spectral Conditions throughout Training","created":"2026-01-04","cited_by_count":0,"area":null},{"id":"2601.14603","title":"Variance-Adaptive Muon: Pre-Orthogonalization Variance Modulation for Efficient Language Model Pretraining","created":"2026-01-21","cited_by_count":0,"area":null},{"id":"2602.13659","title":"Zero-Order Optimization for LLM Fine-Tuning via Learnable Direction Sampling","created":"2026-02-14","cited_by_count":0,"area":null},{"id":"2605.19008","title":"Learn-by-Wire Training Control Governance: Bounded Autonomous Training Under Stress for Stability and Efficiency","created":"2026-05-18","cited_by_count":0,"area":null},{"id":"2605.21104","title":"HORST: Composing Optimizer Geometries for Sparse Transformer Training","created":"2026-05-20","cited_by_count":0,"area":null},{"id":"2605.26327","title":"Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage","created":"2026-05-25","cited_by_count":0,"area":null},{"id":"2606.30634","title":"One-Step Gradient Delay is Not a Barrier for Large-Scale Asynchronous Pipeline Parallel LLM Pretraining","created":"2026-06-29","cited_by_count":0,"area":null},{"id":"2607.04033","title":"OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers","created":"2026-07-04","cited_by_count":0,"area":null},{"id":"2608.28557","title":"Blog: Survey of Optimizers","created":"2026-08-28","cited_by_count":0,"area":null},{"id":"2609.02728","title":"Momentum in large-batch training: Polyak enlarges the critical batch size, Nesterov improves data efficiency","created":"2026-09-02","cited_by_count":0,"area":null},{"id":"2609.33016","title":"Relative Generalization Invariance of LLM Pretraining","created":"2026-09-26","cited_by_count":0,"area":null},{"id":"2609.35297","title":"LionMuon: Alternating Spectral and Sign Descent for Efficient Training","created":"2026-09-28","cited_by_count":0,"area":null},{"id":"2610.03432","title":"OptiSelect: How does the Optimizer Shape Data Curriculum?","created":"2026-10-02","cited_by_count":0,"area":null},{"id":"2610.08975","title":"The Best Optimizer Depends on Batch Size","created":"2026-10-06","cited_by_count":0,"area":null},{"id":"2610.13036","title":"Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf","created":"2026-10-09","cited_by_count":0,"area":null},{"id":"2610.16868","title":"From Parallel to Sequential: Small-Batch Noise Restores Simplicity Bias of Muon","created":"2026-10-12","cited_by_count":0,"area":null},{"id":"2610.16923","title":"Minimax Optimality of Shampoo for Nonstationary Stochastic Optimization","created":"2026-10-12","cited_by_count":0,"area":null},{"id":"2610.17285","title":"Which Geometry on Which Layer? Measured Sign-Spectral Assignment in Mixed-Optimizer Training","created":"2026-10-12","cited_by_count":0,"area":null},{"id":"2610.18695","title":"A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model Training","created":"2026-10-13","cited_by_count":0,"area":null}],"next_cursor":null}