{"total":320,"rows":[{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326,"area":null},{"id":"2404.06395","title":"MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies","created":"2024-04-09","cited_by_count":723,"area":null},{"id":"2401.02954","title":"DeepSeek LLM: Scaling Open-Source Language Models with Longtermism","created":"2024-01-05","cited_by_count":642,"area":"lr"},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288,"area":null},{"id":"2307.10169","title":"Challenges and Applications of Large Language Models","created":"2023-07-19","cited_by_count":254,"area":null},{"id":"2204.13650","title":"Unlocking High-Accuracy Differentially Private Image Classification through Scale","created":"2022-04-28","cited_by_count":224,"area":null},{"id":"2106.06639","title":"Federated Learning with Buffered Asynchronous Aggregation","created":"2021-06-11","cited_by_count":190,"area":null},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184,"area":null},{"id":"2405.18392","title":"Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations","created":"2024-05-28","cited_by_count":170,"area":null},{"id":"2111.10952","title":"ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning","created":"2021-11-22","cited_by_count":161,"area":null},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155,"area":null},{"id":"2106.09647","title":"Deep Learning Through the Lens of Example Difficulty","created":"2021-06-17","cited_by_count":141,"area":null},{"id":"2203.02714","title":"Towards Efficient and Scalable Sharpness-Aware Minimization","created":"2022-03-05","cited_by_count":129,"area":null},{"id":"2303.03329","title":"End-to-End Speech Recognition: A Survey","created":"2023-03-03","cited_by_count":129,"area":null},{"id":"2206.07959","title":"Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?","created":"2022-06-16","cited_by_count":128,"area":null},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121,"area":null},{"id":"2108.01624","title":"Large-Scale Differentially Private BERT","created":"2021-08-03","cited_by_count":118,"area":null},{"id":"2102.02887","title":"Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training","created":"2021-02-04","cited_by_count":110,"area":null},{"id":"2402.03293","title":"Flora: Low-Rank Adapters Are Secretly Gradient Compressors","created":"2024-02-05","cited_by_count":108,"area":null},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93,"area":null},{"id":"2206.07136","title":"Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger","created":"2022-06-14","cited_by_count":88,"area":null},{"id":"2208.06102","title":"Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training","created":"2022-08-12","cited_by_count":79,"area":null},{"id":"2106.07820","title":"On Large-Cohort Training for Federated Learning","created":"2021-06-15","cited_by_count":74,"area":null},{"id":"2102.12470","title":"On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)","created":"2021-02-24","cited_by_count":73,"area":null},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73,"area":null},{"id":"2306.15969","title":"Separable Physics-Informed Neural Networks","created":"2023-06-28","cited_by_count":66,"area":null},{"id":"2403.13101","title":"AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks","created":"2024-03-19","cited_by_count":64,"area":null},{"id":"2106.11872","title":"Randomness In Neural Network Training: Characterizing The Impact of Tooling","created":"2021-06-22","cited_by_count":59,"area":null},{"id":"2206.10654","title":"On the Maximum Hessian Eigenvalue and Generalization","created":"2022-06-21","cited_by_count":57,"area":null},{"id":"2105.07576","title":"Rethinking \"Batch\" in BatchNorm","created":"2021-05-17","cited_by_count":45,"area":null},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44,"area":null},{"id":"2505.13738","title":"Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training","created":"2025-05-19","cited_by_count":43,"area":null},{"id":"2307.06440","title":"No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models","created":"2023-07-12","cited_by_count":41,"area":null},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40,"area":null},{"id":"2412.13337","title":"Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs","created":"2024-12-17","cited_by_count":40,"area":null},{"id":"2209.13083","title":"Why neural networks find simple solutions: the many regularizers of geometric complexity","created":"2022-09-27","cited_by_count":39,"area":null},{"id":"2210.06640","title":"Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities","created":"2022-10-13","cited_by_count":38,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2108.06084","title":"The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models","created":"2021-08-13","cited_by_count":36,"area":null},{"id":"2303.02749","title":"Revisiting the Noise Model of Stochastic Gradient Descent","created":"2023-03-05","cited_by_count":36,"area":null},{"id":"2110.00641","title":"Batch size-invariance for policy optimization","created":"2021-10-01","cited_by_count":35,"area":null},{"id":"2310.02984","title":"Scaling Laws for Associative Memories","created":"2023-10-04","cited_by_count":35,"area":null},{"id":"2502.15938","title":"Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs","created":"2025-02-21","cited_by_count":35,"area":null},{"id":"2210.07681","title":"Quo Vadis: Is Trajectory Forecasting the Key Towards Long-Term Multi-Object Tracking?","created":"2022-10-14","cited_by_count":29,"area":null},{"id":"2408.11029","title":"Scaling Law with Learning Rate Annealing","created":"2024-08-20","cited_by_count":29,"area":null},{"id":"2103.17182","title":"Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization","created":"2021-03-31","cited_by_count":28,"area":null},{"id":"2103.00543","title":"On the Utility of Gradient Compression in Distributed Training Systems","created":"2021-02-28","cited_by_count":26,"area":null},{"id":"2211.07541","title":"Aspects of scaling and scalability for flow-based sampling of lattice QCD","created":"2022-11-14","cited_by_count":26,"area":null},{"id":"2304.03589","title":"On Efficient Training of Large-Scale Deep Learning Models: A Literature Review","created":"2023-04-07","cited_by_count":26,"area":null},{"id":"2205.05474","title":"DeepFilterNet2: Towards Real-Time Speech Enhancement on Embedded Devices for Full-Band Audio","created":"2022-05-11","cited_by_count":24,"area":null},{"id":"2307.13813","title":"How to Scale Your EMA","created":"2023-07-25","cited_by_count":24,"area":null},{"id":"2402.13412","title":"Scaling physics-informed hard constraints with mixture-of-experts","created":"2024-02-20","cited_by_count":24,"area":"moe"},{"id":"2105.14450","title":"Maximizing Parallelism in Distributed Training for Huge Neural Networks","created":"2021-05-30","cited_by_count":23,"area":null},{"id":"2108.10566","title":"sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel Classification","created":"2021-08-24","cited_by_count":23,"area":null},{"id":"2112.05929","title":"Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning","created":"2021-12-11","cited_by_count":23,"area":null},{"id":"2302.02573","title":"Topology-aware Federated Learning in Edge Computing: A Comprehensive Survey","created":"2023-02-06","cited_by_count":23,"area":null},{"id":"2505.23971","title":"Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training","created":"2025-05-29","cited_by_count":23,"area":null},{"id":"2102.08380","title":"Topological Obstructions to Autoencoding","created":"2021-02-16","cited_by_count":20,"area":null},{"id":"2312.00359","title":"Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training","created":"2023-12-01","cited_by_count":20,"area":null},{"id":"2411.17470","title":"Towards Precise Scaling Laws for Video Diffusion Transformers","created":"2024-11-25","cited_by_count":20,"area":"diffusion"},{"id":"2206.01029","title":"Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions","created":"2022-06-02","cited_by_count":19,"area":null},{"id":"2306.09000","title":"When and Why Momentum Accelerates SGD:An Empirical Study","created":"2023-06-15","cited_by_count":19,"area":null},{"id":"2404.19261","title":"High dimensional analysis reveals conservative sharpening and a stochastic edge of stability","created":"2024-04-30","cited_by_count":18,"area":null},{"id":"2405.12807","title":"FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information","created":"2024-05-21","cited_by_count":18,"area":null},{"id":"2506.07254","title":"A Stable Whitening Optimizer for Efficient Neural Network Training","created":"2025-06-08","cited_by_count":18,"area":null},{"id":"2202.00308","title":"PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation","created":"2022-02-01","cited_by_count":17,"area":null},{"id":"2103.00148","title":"Scalable federated machine learning with FEDn","created":"2021-02-27","cited_by_count":16,"area":null},{"id":"2312.12670","title":"On the Role of Server Momentum in Federated Learning","created":"2023-12-19","cited_by_count":16,"area":null},{"id":"2103.12726","title":"Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning","created":"2021-03-23","cited_by_count":15,"area":null},{"id":"2104.07639","title":"Robust Optimization for Multilingual Translation with Imbalanced Data","created":"2021-04-15","cited_by_count":15,"area":null},{"id":"2208.03767","title":"Class-Incremental Learning with Cross-Space Clustering and Controlled Transfer","created":"2022-08-07","cited_by_count":15,"area":null},{"id":"2306.17499","title":"The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks","created":"2023-06-30","cited_by_count":15,"area":null},{"id":"2506.08426","title":"HASFL: Heterogeneity-aware Split Federated Learning over Edge Computing Systems","created":"2025-06-10","cited_by_count":15,"area":null},{"id":"2310.11428","title":"Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression","created":"2023-10-17","cited_by_count":14,"area":null},{"id":"2103.01338","title":"Acceleration via Fractal Learning Rate Schedules","created":"2021-03-01","cited_by_count":13,"area":null},{"id":"2103.06219","title":"Why flatness does and does not correlate with generalization for deep neural networks","created":"2021-03-10","cited_by_count":13,"area":null},{"id":"2112.09245","title":"Automated Deep Learning: Neural Architecture Search Is Not the End","created":"2021-12-16","cited_by_count":13,"area":null},{"id":"2206.07553","title":"On the fast convergence of minibatch heavy ball momentum","created":"2022-06-15","cited_by_count":13,"area":null},{"id":"2106.09204","title":"An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Models","created":"2021-06-17","cited_by_count":12,"area":null},{"id":"2305.12213","title":"Taming Resource Heterogeneity In Distributed ML Training With Dynamic Batching","created":"2023-05-20","cited_by_count":12,"area":null},{"id":"2305.15096","title":"Dynamic Masking Rate Schedules for MLM Pretraining","created":"2023-05-24","cited_by_count":12,"area":null},{"id":"2508.01483","title":"Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler","created":"2025-08-02","cited_by_count":12,"area":null},{"id":"2106.00221","title":"Concurrent Adversarial Learning for Large-Batch Training","created":"2021-06-01","cited_by_count":11,"area":null},{"id":"2112.02663","title":"ES-dRNN: A Hybrid Exponential Smoothing and Dilated Recurrent Neural Network Model for Short-Term Load Forecasting","created":"2021-12-05","cited_by_count":11,"area":null},{"id":"2206.05985","title":"Modeling the Machine Learning Multiverse","created":"2022-06-13","cited_by_count":11,"area":null},{"id":"2212.07295","title":"Maximal Initial Learning Rates in Deep ReLU Networks","created":"2022-12-14","cited_by_count":11,"area":null},{"id":"2303.00586","title":"FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling","created":"2023-03-01","cited_by_count":11,"area":null},{"id":"2409.08770","title":"Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent","created":"2024-09-13","cited_by_count":11,"area":null},{"id":"2102.06025","title":"Large-Scale Training System for 100-Million Classification at Alibaba","created":"2021-02-09","cited_by_count":10,"area":null},{"id":"2111.00856","title":"Large-Scale Deep Learning Optimizations: A Comprehensive Survey","created":"2021-11-01","cited_by_count":10,"area":null},{"id":"2207.00208","title":"e-CLIP: Large-Scale Vision-Language Representation Learning in E-commerce","created":"2022-07-01","cited_by_count":10,"area":null},{"id":"2309.11197","title":"The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute","created":"2023-09-20","cited_by_count":10,"area":null},{"id":"2508.17901","title":"Riemannian Optimization for LoRA on the Stiefel Manifold","created":"2025-08-25","cited_by_count":10,"area":null},{"id":"2110.10165","title":"NAS-HPO-Bench-II: A Benchmark Dataset on Joint Optimization of Convolutional Neural Network Architecture and Training Hyperparameters","created":"2021-10-19","cited_by_count":9,"area":null},{"id":"2203.00089","title":"Amortized Proximal Optimization","created":"2022-02-28","cited_by_count":9,"area":null},{"id":"2305.06435","title":"Phase transitions in the mini-batch size for sparse and dense two-layer neural networks","created":"2023-05-10","cited_by_count":9,"area":null},{"id":"2310.11291","title":"An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent","created":"2023-10-17","cited_by_count":9,"area":null},{"id":"2402.08958","title":"Towards Next-Level Post-Training Quantization of Hyper-Scale Transformers","created":"2024-02-14","cited_by_count":9,"area":null},{"id":"2410.05838","title":"Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit","created":"2024-10-08","cited_by_count":9,"area":null},{"id":"2601.19400","title":"Improved Convergence Rates of Muon Optimizer for Nonconvex Optimization","created":"2026-01-27","cited_by_count":9,"area":null}],"next_cursor":"100"}