{"total":135,"rows":[{"id":"2106.01548","title":"When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations","created":"2021-06-03","cited_by_count":260,"area":null},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184,"area":null},{"id":"2203.02714","title":"Towards Efficient and Scalable Sharpness-Aware Minimization","created":"2022-03-05","cited_by_count":129,"area":null},{"id":"2309.02390","title":"Explaining grokking through circuit efficiency","created":"2023-09-05","cited_by_count":111,"area":null},{"id":"2110.08529","title":"Sharpness-Aware Minimization Improves Language Model Generalization","created":"2021-10-16","cited_by_count":108,"area":null},{"id":"2203.11834","title":"Improving Generalization in Federated Learning by Seeking Flat Minima","created":"2022-03-22","cited_by_count":98,"area":null},{"id":"2102.12470","title":"On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)","created":"2021-02-24","cited_by_count":73,"area":null},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73,"area":null},{"id":"2311.17874","title":"FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information","created":"2023-11-29","cited_by_count":64,"area":null},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63,"area":null},{"id":"2304.05366","title":"The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning","created":"2023-04-11","cited_by_count":63,"area":null},{"id":"2206.10654","title":"On the Maximum Hessian Eigenvalue and Generalization","created":"2022-06-21","cited_by_count":57,"area":null},{"id":"2303.03108","title":"Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization","created":"2023-03-03","cited_by_count":57,"area":null},{"id":"2503.02113","title":"Deep Learning is Not So Mysterious or Different","created":"2025-03-03","cited_by_count":56,"area":null},{"id":"2212.01378","title":"ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning","created":"2022-12-02","cited_by_count":52,"area":null},{"id":"2308.12108","title":"The Local Learning Coefficient: A Singularity-Aware Complexity Measure","created":"2023-08-23","cited_by_count":51,"area":null},{"id":"2505.13738","title":"Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training","created":"2025-05-19","cited_by_count":43,"area":null},{"id":"2210.01620","title":"SAM as an Optimal Relaxation of Bayes","created":"2022-10-04","cited_by_count":41,"area":null},{"id":"2507.07101","title":"Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful","created":"2025-07-09","cited_by_count":41,"area":null},{"id":"2209.13083","title":"Why neural networks find simple solutions: the many regularizers of geometric complexity","created":"2022-09-27","cited_by_count":39,"area":null},{"id":"2302.08982","title":"(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability","created":"2023-02-17","cited_by_count":36,"area":null},{"id":"2110.00641","title":"Batch size-invariance for policy optimization","created":"2021-10-01","cited_by_count":35,"area":null},{"id":"2105.13343","title":"Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error","created":"2021-05-27","cited_by_count":33,"area":null},{"id":"2105.09557","title":"Power-law escape rate of SGD","created":"2021-05-20","cited_by_count":29,"area":null},{"id":"2106.13897","title":"Implicit Gradient Alignment in Distributed and Federated Learning","created":"2021-06-25","cited_by_count":29,"area":null},{"id":"2305.16038","title":"Implicit bias of SGD in $L_{2}$-regularized linear DNNs: One-way jumps from high to low rank","created":"2023-05-25","cited_by_count":29,"area":null},{"id":"2203.03756","title":"Flat minima generalize for low-rank matrix recovery","created":"2022-03-07","cited_by_count":28,"area":null},{"id":"2307.11108","title":"Flatness-Aware Minimization for Domain Generalization","created":"2023-07-20","cited_by_count":27,"area":null},{"id":"2106.02588","title":"Stochastic gradient descent with noise of machine learning type. Part II: Continuous time analysis","created":"2021-06-04","cited_by_count":25,"area":null},{"id":"2302.00849","title":"Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent","created":"2023-02-02","cited_by_count":25,"area":null},{"id":"2206.07252","title":"Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions","created":"2022-06-15","cited_by_count":23,"area":null},{"id":"2409.15156","title":"Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling","created":"2024-09-23","cited_by_count":23,"area":"lr"},{"id":"2504.12532","title":"Generalization through variance: how noise shapes inductive biases in diffusion models","created":"2025-04-16","cited_by_count":23,"area":"diffusion"},{"id":"2602.07852","title":"Emergent Misalignment is Easy, Narrow Misalignment is Hard","created":"2026-02-08","cited_by_count":23,"area":null},{"id":"2405.14578","title":"Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling","created":"2024-05-23","cited_by_count":22,"area":null},{"id":"2312.00359","title":"Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training","created":"2023-12-01","cited_by_count":20,"area":null},{"id":"2306.12230","title":"Fantastic Weights and How to Find Them: Where to Prune in Dynamic Sparse Training","created":"2023-06-21","cited_by_count":17,"area":null},{"id":"2402.16828","title":"Training Neural Networks from Scratch with Parallel Low-Rank Adapters","created":"2024-02-26","cited_by_count":17,"area":null},{"id":"2405.13456","title":"Deep linear networks for regression are implicitly regularized towards flat minima","created":"2024-05-22","cited_by_count":17,"area":null},{"id":"2312.05181","title":"Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections","created":"2023-12-08","cited_by_count":16,"area":null},{"id":"2312.12670","title":"On the Role of Server Momentum in Federated Learning","created":"2023-12-19","cited_by_count":16,"area":null},{"id":"2103.12726","title":"Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning","created":"2021-03-23","cited_by_count":15,"area":null},{"id":"2510.27486","title":"FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Models","created":"2025-10-31","cited_by_count":15,"area":null},{"id":"2105.01747","title":"Information Complexity and Generalization Bounds","created":"2021-05-04","cited_by_count":14,"area":null},{"id":"2505.03793","title":"LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection","created":"2025-05-01","cited_by_count":14,"area":null},{"id":"2604.07666","title":"An Imperfect Verifier is Good Enough: Learning with Noisy Rewards","created":"2026-04-09","cited_by_count":13,"area":null},{"id":"2106.09204","title":"An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Models","created":"2021-06-17","cited_by_count":12,"area":null},{"id":"2301.00008","title":"Effects of Data Geometry in Early Deep Learning","created":"2022-12-29","cited_by_count":12,"area":null},{"id":"2305.12213","title":"Taming Resource Heterogeneity In Distributed ML Training With Dynamic Batching","created":"2023-05-20","cited_by_count":12,"area":null},{"id":"2309.10688","title":"On the different regimes of Stochastic Gradient Descent","created":"2023-09-19","cited_by_count":12,"area":null},{"id":"2104.04874","title":"SGD Implicitly Regularizes Generalization Error","created":"2021-04-10","cited_by_count":10,"area":null},{"id":"2107.09437","title":"Edge of chaos as a guiding principle for modern neural network training","created":"2021-07-20","cited_by_count":10,"area":null},{"id":"2302.09693","title":"mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization","created":"2023-02-19","cited_by_count":10,"area":null},{"id":"2404.03176","title":"Information-Theoretic Generalization Bounds for Deep Neural Networks","created":"2024-04-04","cited_by_count":10,"area":null},{"id":"2203.12330","title":"Predicting the generalization gap in neural networks using topological data analysis","created":"2022-03-23","cited_by_count":9,"area":null},{"id":"2208.07951","title":"On the generalization of learning algorithms that do not converge","created":"2022-08-16","cited_by_count":9,"area":null},{"id":"2212.00261","title":"Task Discovery: Finding the Tasks that Neural Networks Generalize on","created":"2022-12-01","cited_by_count":9,"area":null},{"id":"2301.13703","title":"Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning","created":"2023-01-31","cited_by_count":9,"area":null},{"id":"2410.05838","title":"Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit","created":"2024-10-08","cited_by_count":9,"area":null},{"id":"2506.13234","title":"The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions","created":"2025-06-16","cited_by_count":9,"area":null},{"id":"2402.11215","title":"AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods","created":"2024-02-17","cited_by_count":8,"area":null},{"id":"2510.03871","title":"Optimal Scaling Needs Optimal Norm","created":"2025-10-04","cited_by_count":8,"area":null},{"id":"2407.16427","title":"Stochastic weight matrix dynamics during learning and Dyson Brownian motion","created":"2024-07-23","cited_by_count":7,"area":null},{"id":"2411.00999","title":"Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers","created":"2024-11-01","cited_by_count":7,"area":null},{"id":"2201.09209","title":"Weight Expansion: A New Perspective on Dropout and Generalization","created":"2022-01-23","cited_by_count":6,"area":null},{"id":"2311.02940","title":"The Pursuit of Human Labeling: A New Perspective on Unsupervised Learning","created":"2023-11-06","cited_by_count":6,"area":null},{"id":"2401.11600","title":"Understanding the Generalization Benefits of Late Learning Rate Decay","created":"2024-01-21","cited_by_count":6,"area":null},{"id":"2406.09908","title":"What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?","created":"2024-06-14","cited_by_count":6,"area":null},{"id":"2402.18824","title":"Batch size invariant Adam","created":"2024-02-29","cited_by_count":5,"area":null},{"id":"2312.09323","title":"Perspectives on the State and Future of Deep Learning - 2023","created":"2023-12-07","cited_by_count":4,"area":null},{"id":"2402.01382","title":"Emergence of heavy tails in homogenized stochastic gradient descent","created":"2024-02-02","cited_by_count":4,"area":null},{"id":"2406.00396","title":"Stochastic Resetting Mitigates Latent Gradient Bias of SGD from Label Noise","created":"2024-06-01","cited_by_count":4,"area":null},{"id":"2406.13936","title":"Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods","created":"2024-06-20","cited_by_count":4,"area":null},{"id":"2408.17394","title":"Continual learning with the neural tangent ensemble","created":"2024-08-30","cited_by_count":4,"area":null},{"id":"2412.21124","title":"Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism","created":"2024-12-30","cited_by_count":4,"area":null},{"id":"2505.23489","title":"SGD as Free Energy Minimization: A Thermodynamic View on Neural Network Training","created":"2025-05-29","cited_by_count":4,"area":null},{"id":"2510.26997","title":"Gradient Descent as Loss Landscape Navigation: a Normative Framework for Deriving Learning Rules","created":"2025-10-30","cited_by_count":4,"area":null},{"id":"2604.09258","title":"Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima","created":"2026-04-10","cited_by_count":4,"area":null},{"id":"2605.06152","title":"Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes","created":"2026-05-07","cited_by_count":4,"area":null},{"id":"2112.01433","title":"Loss Landscape Dependent Self-Adjusting Learning Rates in Decentralized Stochastic Gradient Descent","created":"2021-12-02","cited_by_count":3,"area":null},{"id":"2206.00823","title":"Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules","created":"2022-06-02","cited_by_count":3,"area":null},{"id":"2306.02618","title":"Enhance Diffusion to Improve Robust Generalization","created":"2023-06-05","cited_by_count":3,"area":null},{"id":"2605.11491","title":"Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization","created":"2026-05-12","cited_by_count":3,"area":"lr"},{"id":"2306.05300","title":"Anti-Correlated Noise in Epoch-Based Stochastic Gradient Descent: Implications for Weight Variances in Flat Directions","created":"2023-06-08","cited_by_count":2,"area":null},{"id":"2407.19353","title":"Spring-block theory of feature learning in deep neural networks","created":"2024-07-28","cited_by_count":2,"area":null},{"id":"2501.09137","title":"Gradient Descent Converges Linearly to Flatter Minima than Gradient Flow in Shallow Linear Networks","created":"2025-01-15","cited_by_count":2,"area":null},{"id":"2503.11147","title":"Asynchronous Sharpness-Aware Minimization For Fast and Accurate Deep Learning","created":"2025-03-14","cited_by_count":2,"area":null},{"id":"2503.22478","title":"Almost Bayesian: The Fractal Dynamics of Stochastic Gradient Descent","created":"2025-03-28","cited_by_count":2,"area":null},{"id":"2506.12903","title":"Variational Learning Finds Flatter Solutions at the Edge of Stability","created":"2025-06-15","cited_by_count":2,"area":null},{"id":"2507.03545","title":"DOME: Improving Signal-to-Noise in Stochastic Gradient Descent via Sharp-Direction Subspace Filtering","created":"2025-07-04","cited_by_count":2,"area":null},{"id":"2510.04686","title":"How does the optimizer implicitly bias the model merging loss landscape?","created":"2025-10-06","cited_by_count":2,"area":null},{"id":"2601.21624","title":"Training Memory in Deep Neural Networks: Mechanisms, Evidence, and Measurement Gaps","created":"2026-01-29","cited_by_count":2,"area":null},{"id":"2602.05600","title":"On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2605.23778","title":"The physics of AI weather models","created":"2026-05-22","cited_by_count":2,"area":null},{"id":"2204.01365","title":"Deep learning, stochastic gradient descent and diffusion maps","created":"2022-04-04","cited_by_count":1,"area":null},{"id":"2212.13902","title":"Likelihood-based generalization of Markov parameter estimation and multiple shooting objectives in system identification","created":"2022-12-20","cited_by_count":1,"area":null},{"id":"2311.14120","title":"Weight fluctuations in (deep) linear neural networks and a derivation of the inverse-variance flatness relation","created":"2023-11-23","cited_by_count":1,"area":null},{"id":"2406.07983","title":"Meta-Learning Neural Procedural Biases","created":"2024-06-12","cited_by_count":1,"area":null},{"id":"2501.11584","title":"GCSAM: Gradient Centralized Sharpness Aware Minimization","created":"2025-01-20","cited_by_count":1,"area":null},{"id":"2505.24060","title":"Characterising the Inductive Biases of Neural Networks on Boolean Data","created":"2025-05-29","cited_by_count":1,"area":null}],"next_cursor":"100"}