{"total":386,"rows":[{"id":"2304.07193","title":"DINOv2: Learning Robust Visual Features without Supervision","created":"2023-04-14","cited_by_count":7974,"area":null},{"id":"2303.15343","title":"Sigmoid Loss for Language Image Pre-Training","created":"2023-03-27","cited_by_count":3109,"area":null},{"id":"2310.06694","title":"Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning","created":"2023-10-10","cited_by_count":415,"area":null},{"id":"2303.11331","title":"EVA-02: A Visual Representation for Neon Genesis","created":"2023-03-20","cited_by_count":381,"area":null},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265,"area":null},{"id":"2409.20325","title":"Old Optimizer, New Norm: An Anthology","created":"2024-09-30","cited_by_count":193,"area":null},{"id":"2409.11321","title":"SOAP: Improving and Stabilizing Shampoo using Adam","created":"2024-09-17","cited_by_count":192,"area":null},{"id":"2312.03863","title":"Efficient Large Language Models: A Survey","created":"2023-12-06","cited_by_count":186,"area":null},{"id":"2310.00746","title":"RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models","created":"2023-10-01","cited_by_count":175,"area":null},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137,"area":null},{"id":"2402.05862","title":"Let Your Graph Do the Talking: Encoding Structured Data for LLMs","created":"2024-02-08","cited_by_count":118,"area":null},{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114,"area":null},{"id":"2506.15054","title":"Muon Optimizes Under Spectral Norm Constraints","created":"2025-06-18","cited_by_count":86,"area":null},{"id":"2405.20541","title":"Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models","created":"2024-05-30","cited_by_count":84,"area":null},{"id":"2510.05491","title":"NorMuon: Making Muon more efficient and scalable","created":"2025-10-07","cited_by_count":84,"area":null},{"id":"2306.01129","title":"White-Box Transformers via Sparse Rate Reduction","created":"2023-06-01","cited_by_count":80,"area":null},{"id":"2505.02222","title":"Practical Efficiency of Muon for Pretraining","created":"2025-05-04","cited_by_count":80,"area":null},{"id":"2512.24695","title":"Nested Learning: The Illusion of Deep Learning Architectures","created":"2025-12-31","cited_by_count":72,"area":null},{"id":"2504.05295","title":"Dion: Distributed Orthonormalized Updates","created":"2025-04-07","cited_by_count":68,"area":null},{"id":"2507.11005","title":"AdaMuon: Adaptive Muon Optimizer","created":"2025-07-15","cited_by_count":67,"area":null},{"id":"2411.10438","title":"MARS: Unleashing the Power of Variance Reduction for Training Large Models","created":"2024-11-15","cited_by_count":64,"area":null},{"id":"2602.20133","title":"AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization","created":"2026-02-23","cited_by_count":60,"area":null},{"id":"2602.23413","title":"EvoX: Meta-Evolution for Automated Discovery","created":"2026-02-26","cited_by_count":60,"area":null},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2312.10523","title":"Paloma: A Benchmark for Evaluating Language Model Fit","created":"2023-12-16","cited_by_count":54,"area":null},{"id":"2409.03137","title":"The AdEMAMix Optimizer: Better, Faster, Older","created":"2024-09-05","cited_by_count":54,"area":null},{"id":"2305.17212","title":"Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks","created":"2023-05-26","cited_by_count":53,"area":null},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53,"area":null},{"id":"2502.09692","title":"AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers","created":"2025-02-13","cited_by_count":53,"area":null},{"id":"2404.10242","title":"Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology","created":"2024-04-16","cited_by_count":52,"area":null},{"id":"2406.03476","title":"Does your data spark joy? Performance gains from domain upsampling at the end of training","created":"2024-06-05","cited_by_count":52,"area":null},{"id":"2310.05898","title":"Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts","created":"2023-10-09","cited_by_count":51,"area":null},{"id":"2411.16085","title":"Cautious Optimizers: Improving Training with One Line of Code","created":"2024-11-25","cited_by_count":47,"area":null},{"id":"2405.19550","title":"Stress-Testing Capability Elicitation With Password-Locked Models","created":"2024-05-29","cited_by_count":46,"area":null},{"id":"2311.05709","title":"OmniVec: Learning robust representations with cross modal sharing","created":"2023-11-07","cited_by_count":44,"area":null},{"id":"2405.13698","title":"How to set AdamW's weight decay as you scale model and dataset size","created":"2024-05-22","cited_by_count":43,"area":null},{"id":"2310.05900","title":"Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network","created":"2023-10-09","cited_by_count":42,"area":null},{"id":"2501.16371","title":"Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks","created":"2025-01-22","cited_by_count":42,"area":null},{"id":"2307.06440","title":"No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models","created":"2023-07-12","cited_by_count":41,"area":null},{"id":"2407.07972","title":"Deconstructing What Makes a Good Optimizer for Language Models","created":"2024-07-10","cited_by_count":41,"area":null},{"id":"2506.04192","title":"Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise","created":"2025-06-04","cited_by_count":41,"area":null},{"id":"2505.21829","title":"In Search of Adam's Secret Sauce","created":"2025-05-27","cited_by_count":40,"area":null},{"id":"2408.12857","title":"Memory-Efficient LLM Training with Online Subspace Descent","created":"2024-08-23","cited_by_count":39,"area":null},{"id":"2310.17998","title":"Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity","created":"2023-10-27","cited_by_count":38,"area":null},{"id":"2306.13575","title":"Scaling MLPs: A Tale of Inductive Bias","created":"2023-06-23","cited_by_count":37,"area":null},{"id":"2406.10650","title":"The Implicit Bias of Adam on Separable Data","created":"2024-06-15","cited_by_count":36,"area":null},{"id":"2410.19313","title":"COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training","created":"2024-10-25","cited_by_count":33,"area":null},{"id":"2410.20625","title":"LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization","created":"2024-10-27","cited_by_count":32,"area":null},{"id":"2412.04616","title":"Assessing and Learning Alignment of Unimodal Vision and Language Models","created":"2024-12-05","cited_by_count":32,"area":null},{"id":"2509.15816","title":"On the Convergence of Muon and Beyond","created":"2025-09-19","cited_by_count":31,"area":null},{"id":"2401.02020","title":"Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticket","created":"2024-01-04","cited_by_count":30,"area":null},{"id":"2502.17410","title":"COSMOS: A Hybrid Adaptive Optimizer for Memory-Efficient Training of LLMs","created":"2025-02-24","cited_by_count":30,"area":null},{"id":"2502.19002","title":"The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training","created":"2025-02-26","cited_by_count":30,"area":null},{"id":"2411.00446","title":"A Lorentz-Equivariant Transformer for All of the LHC","created":"2024-11-01","cited_by_count":28,"area":null},{"id":"2306.04226","title":"Normalization Layers Are All That Sharpness-Aware Minimization Needs","created":"2023-06-07","cited_by_count":27,"area":null},{"id":"2402.12365","title":"Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators","created":"2024-02-19","cited_by_count":27,"area":null},{"id":"2304.03589","title":"On Efficient Training of Large-Scale Deep Learning Models: A Literature Review","created":"2023-04-07","cited_by_count":26,"area":null},{"id":"2310.09639","title":"DPZero: Private Fine-Tuning of Language Models without Backpropagation","created":"2023-10-14","cited_by_count":26,"area":null},{"id":"2402.01567","title":"Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise","created":"2024-02-02","cited_by_count":26,"area":null},{"id":"2402.03496","title":"Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective","created":"2024-02-05","cited_by_count":25,"area":null},{"id":"2305.07583","title":"MoMo: Momentum Models for Adaptive Learning Rates","created":"2023-05-12","cited_by_count":24,"area":null},{"id":"2308.16271","title":"Emergence of Segmentation with Minimalistic White-Box Transformers","created":"2023-08-30","cited_by_count":24,"area":null},{"id":"2309.01859","title":"NLLB-CLIP -- train performant multilingual image retrieval model on a budget","created":"2023-09-04","cited_by_count":24,"area":null},{"id":"2505.19227","title":"Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law","created":"2025-05-25","cited_by_count":24,"area":null},{"id":"2405.05847","title":"Learned feature representations are biased by complexity, learning order, position, and more","created":"2024-05-09","cited_by_count":23,"area":null},{"id":"2405.14578","title":"Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling","created":"2024-05-23","cited_by_count":22,"area":null},{"id":"2410.23922","title":"Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training","created":"2024-10-31","cited_by_count":22,"area":null},{"id":"2509.14562","title":"LiMuon: Light and Fast Muon Optimizer for Large Models","created":"2025-09-18","cited_by_count":22,"area":null},{"id":"2512.05620","title":"Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales","created":"2025-12-05","cited_by_count":21,"area":null},{"id":"2306.00144","title":"Mechanic: A Learning Rate Tuner","created":"2023-05-31","cited_by_count":20,"area":null},{"id":"2412.13148","title":"SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training","created":"2024-12-17","cited_by_count":20,"area":null},{"id":"2501.13329","title":"Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networks","created":"2025-01-23","cited_by_count":20,"area":null},{"id":"2510.19093","title":"Weight Decay may matter more than muP for Learning Rate Transfer in Practice","created":"2025-10-21","cited_by_count":20,"area":null},{"id":"2602.04940","title":"Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometries","created":"2026-02-04","cited_by_count":20,"area":null},{"id":"2305.05095","title":"Less is More: Removing Text-regions Improves CLIP Training Efficiency and Robustness","created":"2023-05-08","cited_by_count":19,"area":null},{"id":"2306.09000","title":"When and Why Momentum Accelerates SGD:An Empirical Study","created":"2023-06-15","cited_by_count":19,"area":null},{"id":"2505.24749","title":"SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training","created":"2025-05-30","cited_by_count":19,"area":null},{"id":"2404.00438","title":"Communication Efficient Distributed Training with Distributed Lion","created":"2024-03-30","cited_by_count":18,"area":null},{"id":"2404.06012","title":"Diffusion-Based Point Cloud Super-Resolution for mmWave Radar Data","created":"2024-04-09","cited_by_count":17,"area":null},{"id":"2603.09697","title":"Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning","created":"2026-03-10","cited_by_count":17,"area":null},{"id":"2308.10888","title":"Unlocking Accuracy and Fairness in Differentially Private Image Classification","created":"2023-08-21","cited_by_count":16,"area":null},{"id":"2312.02147","title":"Rejuvenating image-GPT as Strong Visual Representation Learners","created":"2023-12-04","cited_by_count":16,"area":null},{"id":"2510.25000","title":"What Really Matters in Matrix-Whitening Optimizers?","created":"2025-10-28","cited_by_count":16,"area":null},{"id":"2510.27403","title":"FedMuon: Accelerating Federated Learning with Matrix Orthogonalization","created":"2025-10-31","cited_by_count":16,"area":null},{"id":"2606.16899","title":"Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization","created":"2026-06-15","cited_by_count":16,"area":null},{"id":"2311.13133","title":"LIMIT: Less Is More for Instruction Tuning Across Evaluation Paradigms","created":"2023-11-22","cited_by_count":14,"area":null},{"id":"2405.16397","title":"AdaFisher: Adaptive Second Order Optimization via Fisher Information","created":"2024-05-26","cited_by_count":14,"area":null},{"id":"2405.20763","title":"Improving Generalization and Convergence by Enhancing Implicit Regularization","created":"2024-05-31","cited_by_count":14,"area":null},{"id":"2406.03348","title":"Position: A Call to Action for a Human-Centered AutoML Paradigm","created":"2024-06-05","cited_by_count":14,"area":null},{"id":"2407.20999","title":"MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning","created":"2024-07-30","cited_by_count":14,"area":null},{"id":"2306.07879","title":"Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity","created":"2023-06-13","cited_by_count":13,"area":null},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13,"area":null},{"id":"2404.01925","title":"Improving Bird's Eye View Semantic Segmentation by Task Decomposition","created":"2024-04-02","cited_by_count":13,"area":null},{"id":"2411.02572","title":"ViTally Consistent: Scaling Biological Representation Learning for Cell Microscopy","created":"2024-11-04","cited_by_count":13,"area":null},{"id":"2506.02281","title":"Angles Don't Lie: Unlocking Training-Efficient RL Through the Model's Own Signals","created":"2025-06-02","cited_by_count":13,"area":null},{"id":"2510.12402","title":"Cautious Weight Decay","created":"2025-10-14","cited_by_count":13,"area":null},{"id":"2403.07684","title":"Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal","created":"2024-03-12","cited_by_count":12,"area":null},{"id":"2408.06697","title":"SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fields","created":"2024-08-13","cited_by_count":12,"area":null},{"id":"2601.23000","title":"Mano: Restriking Manifold Optimization for LLM Training","created":"2026-01-30","cited_by_count":12,"area":null},{"id":"2603.09952","title":"On the Width Scaling of Neural Optimizers Under Matrix Operator Norms I: Row/Column Normalization and Hyperparameter Transfer","created":"2026-03-10","cited_by_count":12,"area":null}],"next_cursor":"100"}