{"total":60,"rows":[{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114,"area":null},{"id":"2202.00661","title":"When Do Flat Minima Optimizers Work?","created":"2022-02-01","cited_by_count":78,"area":null},{"id":"2211.13609","title":"PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization","created":"2022-11-24","cited_by_count":76,"area":null},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68,"area":null},{"id":"2304.05366","title":"The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning","created":"2023-04-11","cited_by_count":63,"area":null},{"id":"2203.08124","title":"Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspective","created":"2022-03-15","cited_by_count":57,"area":null},{"id":"2503.02113","title":"Deep Learning is Not So Mysterious or Different","created":"2025-03-03","cited_by_count":56,"area":null},{"id":"2203.05363","title":"Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)","created":"2022-03-10","cited_by_count":48,"area":null},{"id":"2403.02241","title":"Neural Redshift: Random Networks are not Random Functions","created":"2024-03-04","cited_by_count":43,"area":null},{"id":"2111.12229","title":"Subspace Adversarial Training","created":"2021-11-24","cited_by_count":41,"area":null},{"id":"2507.07101","title":"Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful","created":"2025-07-09","cited_by_count":41,"area":null},{"id":"2210.06441","title":"How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularization","created":"2022-10-12","cited_by_count":40,"area":null},{"id":"2209.13083","title":"Why neural networks find simple solutions: the many regularizers of geometric complexity","created":"2022-09-27","cited_by_count":39,"area":null},{"id":"2306.04251","title":"Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks","created":"2023-06-07","cited_by_count":39,"area":null},{"id":"2302.08982","title":"(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability","created":"2023-02-17","cited_by_count":36,"area":null},{"id":"2307.07873","title":"Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability","created":"2023-07-15","cited_by_count":32,"area":null},{"id":"2306.04815","title":"Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning","created":"2023-06-07","cited_by_count":31,"area":null},{"id":"2204.12446","title":"Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD","created":"2022-04-26","cited_by_count":19,"area":null},{"id":"2110.11749","title":"Feature Learning and Signal Propagation in Deep Neural Networks","created":"2021-10-22","cited_by_count":18,"area":null},{"id":"2210.00706","title":"Information-Theoretic Analysis of Unsupervised Domain Adaptation","created":"2022-10-03","cited_by_count":15,"area":null},{"id":"2502.05300","title":"Parameter Symmetry Potentially Unifies Deep Learning Theory","created":"2025-02-07","cited_by_count":12,"area":null},{"id":"2211.10691","title":"Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States","created":"2022-11-19","cited_by_count":11,"area":null},{"id":"2302.01952","title":"On a continuous time model of gradient descent dynamics and instability in deep learning","created":"2023-02-03","cited_by_count":11,"area":null},{"id":"2302.09693","title":"mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization","created":"2023-02-19","cited_by_count":10,"area":null},{"id":"2306.08590","title":"Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning","created":"2023-06-14","cited_by_count":10,"area":null},{"id":"2205.11361","title":"Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent","created":"2022-05-23","cited_by_count":9,"area":null},{"id":"2505.12387","title":"Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning","created":"2025-05-18","cited_by_count":9,"area":null},{"id":"2202.13733","title":"On the Benefits of Large Learning Rates for Kernel Methods","created":"2022-02-28","cited_by_count":8,"area":null},{"id":"2209.10080","title":"Deep Double Descent via Smooth Interpolation","created":"2022-09-21","cited_by_count":8,"area":null},{"id":"2303.17940","title":"Per-Example Gradient Regularization Improves Learning Signals from Noisy Data","created":"2023-03-31","cited_by_count":7,"area":null},{"id":"2404.05782","title":"Dynamical stability and chaos in artificial neural network trajectories along training","created":"2024-04-08","cited_by_count":6,"area":null},{"id":"2111.11542","title":"Depth Without the Magic: Inductive Bias of Natural Gradient Descent","created":"2021-11-22","cited_by_count":5,"area":null},{"id":"2305.02247","title":"Select without Fear: Almost All Mini-Batch Schedules Generalize Optimally","created":"2023-05-03","cited_by_count":5,"area":null},{"id":"2306.10598","title":"DropCompute: simple and more robust distributed synchronous training via compute variance reduction","created":"2023-06-18","cited_by_count":4,"area":null},{"id":"2312.09323","title":"Perspectives on the State and Future of Deep Learning - 2023","created":"2023-12-07","cited_by_count":4,"area":null},{"id":"2405.15706","title":"The Impact of Geometric Complexity on Neural Collapse in Transfer Learning","created":"2024-05-24","cited_by_count":4,"area":null},{"id":"2505.23489","title":"SGD as Free Energy Minimization: A Thermodynamic View on Neural Network Training","created":"2025-05-29","cited_by_count":4,"area":null},{"id":"2111.00841","title":"Free Probability for predicting the performance of feed-forward fully connected neural networks","created":"2021-11-01","cited_by_count":3,"area":null},{"id":"2406.11463","title":"Just How Flexible are Neural Networks in Practice?","created":"2024-06-17","cited_by_count":3,"area":null},{"id":"2211.15853","title":"Disentangling the Mechanisms Behind Implicit Regularization in SGD","created":"2022-11-29","cited_by_count":2,"area":null},{"id":"2302.02971","title":"U-Clip: On-Average Unbiased Stochastic Gradient Clipping","created":"2023-02-06","cited_by_count":2,"area":null},{"id":"2306.05300","title":"Anti-Correlated Noise in Epoch-Based Stochastic Gradient Descent: Implications for Weight Variances in Flat Directions","created":"2023-06-08","cited_by_count":2,"area":null},{"id":"2307.05789","title":"Implicit regularisation in stochastic gradient descent: from single-objective to two-player games","created":"2023-07-11","cited_by_count":2,"area":null},{"id":"2407.03848","title":"Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks","created":"2024-07-04","cited_by_count":2,"area":null},{"id":"2408.14677","title":"Can Optimization Trajectories Explain Multi-Task Transfer?","created":"2024-08-26","cited_by_count":2,"area":null},{"id":"2505.19087","title":"Temperature is All You Need for Generalization in Langevin Dynamics and other Markov Processes","created":"2025-05-25","cited_by_count":2,"area":null},{"id":"2207.07654","title":"Learning inducing points and uncertainty on molecular data by scalable variational Gaussian processes","created":"2022-07-16","cited_by_count":1,"area":null},{"id":"2302.05608","title":"Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis","created":"2023-02-11","cited_by_count":1,"area":null},{"id":"2306.09991","title":"Evolutionary Algorithms in the Light of SGD: Limit Equivalence, Minima Flatness, and Transfer Learning","created":"2023-05-20","cited_by_count":1,"area":null},{"id":"2504.18207","title":"Gradient Descent as a Shrinkage Operator for Spectral Bias","created":"2025-04-25","cited_by_count":1,"area":null},{"id":"2508.15086","title":"Wormhole Dynamics in Deep Neural Networks","created":"2025-08-20","cited_by_count":1,"area":null},{"id":"2510.25800","title":"FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks","created":"2025-10-29","cited_by_count":1,"area":null},{"id":"2606.31282","title":"Revisiting the Volume Hypothesis","created":"2026-06-30","cited_by_count":1,"area":null},{"id":"2203.08174","title":"Towards understanding deep learning with the natural clustering prior","created":"2022-03-15","cited_by_count":0,"area":null},{"id":"2208.08085","title":"Detection and Mitigation of Byzantine Attacks in Distributed Training","created":"2022-08-17","cited_by_count":0,"area":null},{"id":"2406.03696","title":"Error dynamics of mini-batch gradient descent with random reshuffling for least squares regression","created":"2024-06-06","cited_by_count":0,"area":null},{"id":"2503.03139","title":"Convergence Analysis of Federated Learning Methods Using Backward Error Analysis","created":"2025-03-05","cited_by_count":0,"area":null},{"id":"2505.24469","title":"Smooth Model Compression without Fine-Tuning","created":"2025-05-30","cited_by_count":0,"area":null},{"id":"2604.06366","title":"Stochastic Gradient Descent in the Saddle-to-Saddle Regime of Deep Linear Networks","created":"2026-04-07","cited_by_count":0,"area":null},{"id":"2604.26496","title":"Robust Alignment: Harmonizing Clean Accuracy and Adversarial Robustness in Adversarial Training","created":"2026-04-29","cited_by_count":0,"area":null}],"next_cursor":null}