{"total":97,"rows":[{"id":"2404.06395","title":"MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies","created":"2024-04-09","cited_by_count":723,"area":null},{"id":"2402.12354","title":"LoRA+: Efficient Low Rank Adaptation of Large Models","created":"2024-02-19","cited_by_count":377,"area":null},{"id":"2403.17887","title":"The Unreasonable Ineffectiveness of the Deeper Layers","created":"2024-03-26","cited_by_count":226,"area":"lr"},{"id":"2510.05491","title":"NorMuon: Making Muon more efficient and scalable","created":"2025-10-07","cited_by_count":84,"area":null},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73,"area":null},{"id":"2405.14813","title":"Scalable Optimization in the Modular Norm","created":"2024-05-23","cited_by_count":68,"area":null},{"id":"2502.05795","title":"The Curse of Depth in Large Language Models","created":"2025-02-09","cited_by_count":62,"area":null},{"id":"2406.08447","title":"The Impact of Initialization on LoRA Finetuning Dynamics","created":"2024-06-12","cited_by_count":54,"area":null},{"id":"2312.16903","title":"Spike No More: Stabilizing the Pre-training of Large Language Models","created":"2023-12-28","cited_by_count":52,"area":null},{"id":"2412.13795","title":"Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN","created":"2024-12-18","cited_by_count":37,"area":null},{"id":"2502.02732","title":"Peri-LN: Revisiting Normalization Layer in the Transformer Architecture","created":"2025-02-04","cited_by_count":34,"area":null},{"id":"2503.04715","title":"Predictable Scale: Part I, Step Law -- Optimal Hyperparameter Scaling Law in Large Language Model Pretraining","created":"2025-03-06","cited_by_count":34,"area":null},{"id":"2407.17465","title":"u-$\\mu$P: The Unit-Scaled Maximal Update Parametrization","created":"2024-07-24","cited_by_count":29,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2406.06248","title":"Compute Better Spent: Replacing Dense Layers with Structured Matrices","created":"2024-06-10","cited_by_count":28,"area":null},{"id":"2408.13359","title":"Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler","created":"2024-08-23","cited_by_count":28,"area":null},{"id":"2512.22382","title":"Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration","created":"2025-12-26","cited_by_count":27,"area":null},{"id":"2410.05661","title":"Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models","created":"2024-10-08","cited_by_count":25,"area":"moe"},{"id":"2402.17457","title":"Super Consistency of Neural Network Landscapes and Learning Rate Transfer","created":"2024-02-27","cited_by_count":23,"area":null},{"id":"2409.15156","title":"Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling","created":"2024-09-23","cited_by_count":23,"area":"lr"},{"id":"2502.06857","title":"Gemstones: A Model Suite for Multi-Faceted Scaling Laws","created":"2025-02-07","cited_by_count":20,"area":null},{"id":"2510.19093","title":"Weight Decay may matter more than muP for Learning Rate Transfer in Practice","created":"2025-10-21","cited_by_count":20,"area":null},{"id":"2502.02531","title":"Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer","created":"2025-02-04","cited_by_count":19,"area":null},{"id":"2405.13456","title":"Deep linear networks for regression are implicitly regularized towards flat minima","created":"2024-05-22","cited_by_count":17,"area":null},{"id":"2509.10167","title":"The Hidden Width of Deep ResNets: Tight Error Bounds and Phase Diagram","created":"2025-09-12","cited_by_count":17,"area":null},{"id":"2505.23194","title":"Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics","created":"2025-05-29","cited_by_count":16,"area":null},{"id":"2410.04642","title":"The Optimization Landscape of SGD Across the Feature Learning Strength","created":"2024-10-06","cited_by_count":13,"area":null},{"id":"2601.20205","title":"Hyperparameter Transfer with Mixture-of-Expert Layers","created":"2026-01-28","cited_by_count":13,"area":"moe"},{"id":"2603.28743","title":"Rethinking Language Model Scaling under Transferable Hypersphere Optimization","created":"2026-03-30","cited_by_count":12,"area":"lr"},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2412.11521","title":"On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory","created":"2024-12-16","cited_by_count":10,"area":null},{"id":"2410.02247","title":"Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization","created":"2024-10-03","cited_by_count":9,"area":null},{"id":"2410.05838","title":"Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit","created":"2024-10-08","cited_by_count":9,"area":null},{"id":"2509.25087","title":"Scaling with Collapse: Efficient and Predictable Training of LLM Families","created":"2025-09-29","cited_by_count":9,"area":null},{"id":"2602.06204","title":"Learning Rate Scaling across LoRA Ranks and Transfer to Full Finetuning","created":"2026-02-05","cited_by_count":9,"area":null},{"id":"2602.21341","title":"Scaling View Synthesis Transformers","created":"2026-02-24","cited_by_count":9,"area":null},{"id":"2510.03871","title":"Optimal Scaling Needs Optimal Norm","created":"2025-10-04","cited_by_count":8,"area":null},{"id":"2510.15262","title":"Robust Layerwise Scaling Rules by Proper Weight Decay Tuning","created":"2025-10-17","cited_by_count":8,"area":null},{"id":"2601.21942","title":"Clustering in Deep Stochastic Transformers","created":"2026-01-29","cited_by_count":8,"area":null},{"id":"2406.00153","title":"$\\mu$LO: Compute-Efficient Meta-Generalization of Learned Optimizers","created":"2024-05-31","cited_by_count":7,"area":null},{"id":"2407.04117","title":"Predictive Coding Networks and Inference Learning: Tutorial and Survey","created":"2024-07-04","cited_by_count":7,"area":null},{"id":"2512.22768","title":"Understanding the Mechanisms of Fast Hyperparameter Transfer","created":"2025-12-28","cited_by_count":7,"area":null},{"id":"2607.13491","title":"DeepLoop: Depth Scaling for Looped Transformers","created":"2026-07-15","cited_by_count":7,"area":null},{"id":"2505.15270","title":"Scaling Diffusion Transformers Efficiently via $\\mu$P","created":"2025-05-21","cited_by_count":6,"area":"diffusion"},{"id":"2506.15025","title":"Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size","created":"2025-06-17","cited_by_count":6,"area":null},{"id":"2506.16884","title":"The Importance of Being Lazy: Scaling Limits of Continual Learning","created":"2025-06-20","cited_by_count":6,"area":null},{"id":"2601.13308","title":"Scaling laws for amplitude surrogates","created":"2026-01-19","cited_by_count":6,"area":null},{"id":"2602.21472","title":"The Design Space of Tri-Modal Masked Diffusion Models","created":"2026-02-25","cited_by_count":6,"area":"diffusion"},{"id":"2502.05967","title":"$\\mu$nit Scaling: Simple and Scalable FP8 LLM Training","created":"2025-02-09","cited_by_count":5,"area":null},{"id":"2506.00846","title":"Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs","created":"2025-06-01","cited_by_count":5,"area":null},{"id":"2601.05049","title":"How to Set the Learning Rate for Large-Scale Pre-training?","created":"2026-01-08","cited_by_count":5,"area":"moe"},{"id":"2608.08888","title":"Full-bandwidth transformer","created":"2026-08-09","cited_by_count":5,"area":null},{"id":"2502.17405","title":"Function-Space Learning Rates","created":"2025-02-24","cited_by_count":4,"area":null},{"id":"2503.09565","title":"Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $\\mu$P Parametrization","created":"2025-03-12","cited_by_count":4,"area":null},{"id":"2508.09752","title":"$\\mu$-Parametrization for Mixture of Experts","created":"2025-08-13","cited_by_count":3,"area":"moe"},{"id":"2511.01734","title":"A Proof of Learning Rate Transfer under $\\mu$P","created":"2025-11-03","cited_by_count":3,"area":null},{"id":"2604.27077","title":"Learning Rate Transfer in Normalized Transformers","created":"2026-04-29","cited_by_count":3,"area":null},{"id":"2606.25971","title":"Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors","created":"2026-06-24","cited_by_count":3,"area":"moe"},{"id":"2507.05566","title":"SingLoRA: Low Rank Adaptation Using a Single Matrix","created":"2025-07-08","cited_by_count":2,"area":null},{"id":"2601.04890","title":"Learnable Multipliers: Freeing the Scale of Language Model Matrix Layers","created":"2026-01-08","cited_by_count":2,"area":null},{"id":"2602.07697","title":"On the Infinite Width and Depth Limits of Predictive Coding Networks","created":"2026-02-07","cited_by_count":2,"area":null},{"id":"2604.01577","title":"Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization","created":"2026-04-02","cited_by_count":2,"area":null},{"id":"2605.21486","title":"Quantifying Hyperparameter Transfer and the Importance of Embedding Layer Learning Rate","created":"2026-05-20","cited_by_count":2,"area":null},{"id":"2606.20299","title":"Statistical Properties of Training & Generalization","created":"2026-06-18","cited_by_count":2,"area":null},{"id":"2607.14018","title":"Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth","created":"2026-07-15","cited_by_count":2,"area":null},{"id":"2607.23777","title":"Scale Weight Decay and Train Better","created":"2026-07-26","cited_by_count":2,"area":null},{"id":"2608.11859","title":"Small-Scale Experiments: Are We There Yet?","created":"2026-08-12","cited_by_count":2,"area":null},{"id":"2505.16260","title":"Small-to-Large Generalization: Data Influences Models Consistently Across Scale","created":"2025-05-22","cited_by_count":1,"area":null},{"id":"2506.19396","title":"Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators","created":"2025-06-24","cited_by_count":1,"area":null},{"id":"2512.13705","title":"Scaling and Transferability of Annealing Strategies in Large Language Model Training","created":"2025-12-05","cited_by_count":1,"area":"moe"},{"id":"2605.07815","title":"OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling","created":"2026-05-08","cited_by_count":1,"area":null},{"id":"2605.15290","title":"GQA-{\\mu}P: The maximal parameterization update for grouped query attention","created":"2026-05-14","cited_by_count":1,"area":null},{"id":"2606.00340","title":"Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks","created":"2026-05-29","cited_by_count":1,"area":null},{"id":"2606.18246","title":"Variable-Width Transformers","created":"2026-06-16","cited_by_count":1,"area":null},{"id":"2607.27731","title":"Towards joint scaling laws with optimal batch size schedules","created":"2026-07-30","cited_by_count":1,"area":null},{"id":"2408.08681","title":"A Mean Field Ansatz for Zero-Shot Weight Transfer","created":"2024-08-16","cited_by_count":0,"area":null},{"id":"2410.03972","title":"Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks","created":"2024-10-04","cited_by_count":0,"area":null},{"id":"2412.19638","title":"Xmodel-2 Technical Report","created":"2024-12-27","cited_by_count":0,"area":"lr"},{"id":"2503.23982","title":"Deep Neural Nets as Hamiltonians","created":"2025-03-31","cited_by_count":0,"area":null},{"id":"2506.08885","title":"AdversariaL attacK sAfety aLIgnment(ALKALI): Safeguarding LLMs through GRACE: Geometric Representation-Aware Contrastive Enhancement- Introducing Adversarial Vulnerability Quality Index (AVQI)","created":"2025-06-10","cited_by_count":0,"area":"alignment"},{"id":"2509.00362","title":"Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks","created":"2025-08-30","cited_by_count":0,"area":null},{"id":"2509.12581","title":"Exploring Training Data Attribution under Limited Access Constraints","created":"2025-09-16","cited_by_count":0,"area":null},{"id":"2511.19496","title":"Xmodel-2.5: 1.3B Data-Efficient Reasoning SLM","created":"2025-11-23","cited_by_count":0,"area":"lr"},{"id":"2512.21075","title":"Feature Learning Dynamics in Infinite-Depth Neural Networks","created":"2025-12-24","cited_by_count":0,"area":null},{"id":"2601.01306","title":"Towards a Principled Muon under $\\mu\\mathsf{P}$: Ensuring Spectral Conditions throughout Training","created":"2026-01-04","cited_by_count":0,"area":null},{"id":"2602.07378","title":"Dichotomy of Feature Learning and Unlearning: Fast-Slow Analysis on Neural Networks with Stochastic Gradient Descent","created":"2026-02-07","cited_by_count":0,"area":"safety"},{"id":"2602.20937","title":"Extending $\\mu$P: Spectral Conditions for Feature Learning Across Optimizers","created":"2026-02-24","cited_by_count":0,"area":null},{"id":"2606.04048","title":"Unlocking Feature Learning in Gated Delta Networks at Scale","created":"2026-06-02","cited_by_count":0,"area":null},{"id":"2606.05610","title":"Predictable Scaling Laws of Optimal Hyperparameters for LLM Continued Pre-training","created":"2026-06-04","cited_by_count":0,"area":null},{"id":"2606.30831","title":"Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product","created":"2026-06-29","cited_by_count":0,"area":null},{"id":"2607.05019","title":"Beyond Modality Fusion: Deep Ensembles for Multimodal Classification","created":"2026-07-06","cited_by_count":0,"area":null},{"id":"2607.07884","title":"Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks","created":"2026-07-08","cited_by_count":0,"area":null},{"id":"2609.00686","title":"A Study of Hidden-State Optimization Order in Predictive Coding Networks","created":"2026-09-01","cited_by_count":0,"area":null},{"id":"2609.01657","title":"NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference","created":"2026-08-31","cited_by_count":0,"area":"mllm"},{"id":"2609.08690","title":"Hyperparameter Scaling Laws Across MoE Sparsity","created":"2026-09-08","cited_by_count":0,"area":"moe"},{"id":"2609.32060","title":"Depth Laws for the Precision Floor of Trained Neural Networks: Amplification, Residual Scaling, and a Quantization-Aware Training Paradox","created":"2026-09-25","cited_by_count":0,"area":null},{"id":"2610.14344","title":"On the Depth Transfer of Neural Optimizers via L-Smoothness","created":"2026-10-09","cited_by_count":0,"area":null}],"next_cursor":null}