{"list":"other","total":54,"rows":[{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2006.11239","title":"Denoising Diffusion Probabilistic Models","created":"2020-06-19","cited_by_count":21076},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"1905.00414","title":"Similarity of Neural Network Representations Revisited","created":"2019-05-01","cited_by_count":1755},{"id":"1312.6120","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","created":"2013-12-20","cited_by_count":763},{"id":"1902.06720","title":"Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent","created":"2019-02-18","cited_by_count":738},{"id":"1812.07956","title":"On Lazy Training in Differentiable Programming","created":"2018-12-19","cited_by_count":675},{"id":"1802.04434","title":"signSGD: Compressed Optimisation for Non-Convex Problems","created":"2018-02-13","cited_by_count":571},{"id":"1804.06561","title":"A Mean Field View of the Landscape of Two-Layers Neural Networks","created":"2018-04-18","cited_by_count":466},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1805.09545","title":"On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport","created":"2018-05-24","cited_by_count":426},{"id":"2305.16264","title":"Scaling Data-Constrained Language Models","created":"2023-05-25","cited_by_count":403},{"id":"2002.09277","title":"Kernel and Rich Regimes in Overparametrized Models","created":"2020-02-20","cited_by_count":306},{"id":"1912.07145","title":"PyHessian: Neural Networks Through the Lens of the Hessian","created":"2019-12-16","cited_by_count":292},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"1902.06015","title":"Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit","created":"2019-02-16","cited_by_count":187},{"id":"2010.15110","title":"Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel","created":"2020-10-28","cited_by_count":187},{"id":"2007.15801","title":"Finite Versus Infinite Neural Networks: an Empirical Study","created":"2020-07-31","cited_by_count":157},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155},{"id":"2402.01092","title":"A Dynamical Model of Neural Scaling Laws","created":"2024-02-02","cited_by_count":111},{"id":"2106.09524","title":"Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity","created":"2021-06-17","cited_by_count":110},{"id":"2310.17813","title":"A Spectral Condition for Feature Learning","created":"2023-10-26","cited_by_count":109},{"id":"2310.06110","title":"Grokking as the Transition from Lazy to Rich Training Dynamics","created":"2023-10-09","cited_by_count":108},{"id":"2111.00034","title":"Neural Networks as Kernel Learners: The Silent Alignment Effect","created":"2021-10-29","cited_by_count":106},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105},{"id":"2310.02244","title":"Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks","created":"2023-10-03","cited_by_count":97},{"id":"2106.15933","title":"Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity","created":"2021-06-30","cited_by_count":85},{"id":"2006.15081","title":"On the Generalization Benefit of Noise in Stochastic Gradient Descent","created":"2020-06-26","cited_by_count":82},{"id":"1805.10074","title":"Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes","created":"2018-05-25","cited_by_count":69},{"id":"2405.15074","title":"4+3 Phases of Compute-Optimal Neural Scaling Laws","created":"2024-05-23","cited_by_count":68},{"id":"2409.17858","title":"How Feature Learning Can Improve Neural Scaling Laws","created":"2024-09-26","cited_by_count":56},{"id":"2211.11567","title":"Neural networks trained with SGD learn distributions of increasing complexity","created":"2022-11-21","cited_by_count":50},{"id":"2304.03408","title":"Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks","created":"2023-04-06","cited_by_count":43},{"id":"2305.18411","title":"Feature-Learning Networks Are Consistent Across Widths At Realistic Scales","created":"2023-05-28","cited_by_count":41},{"id":"2210.04860","title":"Second-order regression models exhibit progressive sharpening to the edge of stability","created":"2022-10-10","cited_by_count":40},{"id":"2405.15712","title":"Infinite Limits of Multi-head Transformer Dynamics","created":"2024-05-24","cited_by_count":39},{"id":"2001.07301","title":"On the infinite width limit of neural networks with a standard parameterization","created":"2020-01-21","cited_by_count":38},{"id":"2308.01814","title":"Tensor Programs IVb: Adaptive Optimization in the Infinite-Width Limit","created":"2023-08-03","cited_by_count":38},{"id":"2303.15438","title":"On the Stepwise Nature of Self-Supervised Learning","created":"2023-03-27","cited_by_count":36},{"id":"2206.10012","title":"Limitations of the NTK for Understanding Generalization in Deep Learning","created":"2022-06-20","cited_by_count":35},{"id":"2010.07344","title":"Temperature check: theory and practice for training models with softmax-cross-entropy losses","created":"2020-10-14","cited_by_count":31},{"id":"2206.12314","title":"Learning sparse features can lead to overfitting in neural networks","created":"2022-06-24","cited_by_count":23},{"id":"2402.17457","title":"Super Consistency of Neural Network Landscapes and Learning Rate Transfer","created":"2024-02-27","cited_by_count":23},{"id":"2311.02076","title":"Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos","created":"2023-11-03","cited_by_count":22},{"id":"2302.12250","title":"Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width","created":"2023-02-23","cited_by_count":19},{"id":"2405.17580","title":"Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes","created":"2024-05-27","cited_by_count":18},{"id":"2405.13456","title":"Deep linear networks for regression are implicitly regularized towards flat minima","created":"2024-05-22","cited_by_count":17},{"id":"2406.05033","title":"Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes","created":"2024-06-07","cited_by_count":14},{"id":"2309.10688","title":"On the different regimes of Stochastic Gradient Descent","created":"2023-09-19","cited_by_count":12},{"id":"2306.08590","title":"Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning","created":"2023-06-14","cited_by_count":10},{"id":"2301.13703","title":"Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning","created":"2023-01-31","cited_by_count":9}],"next_cursor":null}