{"total":75,"rows":[{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154,"area":null},{"id":"2110.01654","title":"Improved architectures and training algorithms for deep operator networks","created":"2021-10-04","cited_by_count":86,"area":null},{"id":"2309.06497","title":"A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale","created":"2023-09-12","cited_by_count":77,"area":null},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62,"area":null},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45,"area":null},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44,"area":null},{"id":"2110.04622","title":"Does Preprocessing Help Training Over-parameterized Neural Networks?","created":"2021-10-09","cited_by_count":32,"area":null},{"id":"2201.12250","title":"Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization","created":"2022-01-28","cited_by_count":30,"area":null},{"id":"2106.07887","title":"Credit Assignment in Neural Networks through Deep Feedback Control","created":"2021-06-15","cited_by_count":29,"area":null},{"id":"2211.14227","title":"Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing","created":"2022-11-25","cited_by_count":29,"area":null},{"id":"2306.02295","title":"A Mathematical Abstraction for Balancing the Trade-off Between Creativity and Reality in Large Language Models","created":"2023-06-04","cited_by_count":29,"area":null},{"id":"2203.04248","title":"Dual Lottery Ticket Hypothesis","created":"2022-03-08","cited_by_count":27,"area":null},{"id":"2308.12247","title":"How to Protect Copyright Data in Optimization of Large Language Models?","created":"2023-08-23","cited_by_count":26,"area":null},{"id":"2401.10190","title":"A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions","created":"2024-01-18","cited_by_count":26,"area":null},{"id":"2106.04881","title":"Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms","created":"2021-06-09","cited_by_count":23,"area":null},{"id":"2208.04508","title":"Training Overparametrized Neural Networks in Sublinear Time","created":"2022-08-09","cited_by_count":22,"area":null},{"id":"2402.03864","title":"The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2107.01739","title":"KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks","created":"2021-07-04","cited_by_count":20,"area":null},{"id":"2206.00833","title":"Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm","created":"2022-06-02","cited_by_count":20,"area":null},{"id":"2310.03597","title":"Sampling via Gradient Flows in the Space of Probability Measures","created":"2023-10-05","cited_by_count":20,"area":null},{"id":"2302.07384","title":"The Geometry of Neural Nets' Parameter Spaces Under Reparametrization","created":"2023-02-14","cited_by_count":16,"area":null},{"id":"2302.11024","title":"Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance","created":"2023-02-21","cited_by_count":16,"area":null},{"id":"2312.12226","title":"On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width","created":"2023-12-19","cited_by_count":15,"area":null},{"id":"2406.06420","title":"An Improved Empirical Fisher Approximation for Natural Gradient Descent","created":"2024-06-10","cited_by_count":15,"area":null},{"id":"2109.14158","title":"Second-Order Neural ODE Optimizer","created":"2021-09-29","cited_by_count":11,"area":null},{"id":"2206.15273","title":"Invariance Properties of the Natural Gradient in Overparametrised Systems","created":"2022-06-30","cited_by_count":11,"area":null},{"id":"2307.06565","title":"Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification","created":"2023-07-13","cited_by_count":11,"area":null},{"id":"2108.04620","title":"A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions","created":"2021-08-10","cited_by_count":10,"area":null},{"id":"2108.08106","title":"Existence, uniqueness, and convergence rates for gradient flows in the training of artificial neural networks with ReLU activation","created":"2021-08-18","cited_by_count":9,"area":null},{"id":"2112.07369","title":"Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions","created":"2021-12-13","cited_by_count":9,"area":null},{"id":"2203.00089","title":"Amortized Proximal Optimization","created":"2022-02-28","cited_by_count":9,"area":null},{"id":"2307.14531","title":"Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum","created":"2023-07-26","cited_by_count":9,"area":null},{"id":"2310.11291","title":"An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent","created":"2023-10-17","cited_by_count":9,"area":null},{"id":"2406.17263","title":"Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows","created":"2024-06-25","cited_by_count":9,"area":null},{"id":"2201.06656","title":"Generalization in Supervised Learning Through Riemannian Contraction","created":"2022-01-17","cited_by_count":8,"area":null},{"id":"2202.04124","title":"A Mini-Block Fisher Method for Deep Neural Networks","created":"2022-02-08","cited_by_count":8,"area":null},{"id":"2303.05473","title":"Natural Gradient Methods: Perspectives, Efficient-Scalable Approximations, and Analysis","created":"2023-03-06","cited_by_count":8,"area":null},{"id":"2406.01870","title":"Understanding Stochastic Natural Gradient Variational Inference","created":"2024-06-04","cited_by_count":8,"area":null},{"id":"2106.03947","title":"TENGraD: Time-Efficient Natural Gradient Descent with Exact Fisher-Block Inversion","created":"2021-06-07","cited_by_count":7,"area":null},{"id":"2202.06236","title":"Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems","created":"2022-02-13","cited_by_count":7,"area":null},{"id":"2206.15143","title":"Scalable K-FAC Training for Deep Neural Networks with Distributed Preconditioning","created":"2022-06-30","cited_by_count":7,"area":null},{"id":"2302.02904","title":"Rethinking Gauss-Newton for learning over-parameterized models","created":"2023-02-06","cited_by_count":6,"area":null},{"id":"2310.00137","title":"On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective","created":"2023-09-29","cited_by_count":6,"area":null},{"id":"2207.07287","title":"Riemannian Natural Gradient Methods","created":"2022-07-15","cited_by_count":5,"area":null},{"id":"2305.15912","title":"Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks","created":"2023-05-25","cited_by_count":5,"area":null},{"id":"2111.14039","title":"Generalization Performance of Empirical Risk Minimization on Over-parameterized Deep ReLU Nets","created":"2021-11-28","cited_by_count":4,"area":null},{"id":"2404.10771","title":"TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision","created":"2024-04-16","cited_by_count":4,"area":null},{"id":"2112.07344","title":"SCORE: Approximating Curvature Information under Self-Concordant Regularization","created":"2021-12-14","cited_by_count":3,"area":null},{"id":"2206.04663","title":"Provably efficient variational generative modeling of quantum many-body systems via quantum-probabilistic information geometry","created":"2022-06-09","cited_by_count":3,"area":null},{"id":"2312.16609","title":"Exploiting hidden structures in non-convex games for convergence to Nash equilibrium","created":"2023-12-27","cited_by_count":3,"area":null},{"id":"2404.14875","title":"Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks","created":"2024-04-23","cited_by_count":3,"area":null},{"id":"2412.14031","title":"A Riemannian Optimization Perspective of the Gauss-Newton Method for Feedforward Neural Networks","created":"2024-12-18","cited_by_count":3,"area":null},{"id":"2111.15317","title":"AutoDrop: Training Deep Learning Models with Automatic Learning Rate Drop","created":"2021-11-30","cited_by_count":2,"area":null},{"id":"2202.06232","title":"Generalized Tangent Kernel: A Unified Geometric Foundation for Natural Gradient and Standard Gradient","created":"2022-02-13","cited_by_count":2,"area":null},{"id":"2506.08464","title":"MAC: An Efficient Gradient Preconditioning using Mean Activation Approximated Curvature","created":"2025-06-10","cited_by_count":2,"area":null},{"id":"2110.10017","title":"Neural Network Compatible Off-Policy Natural Actor-Critic Algorithm","created":"2021-10-19","cited_by_count":1,"area":null},{"id":"2312.03885","title":"Gathering and Exploiting Higher-Order Information when Training Large Structured Models","created":"2023-12-06","cited_by_count":1,"area":null},{"id":"2505.24592","title":"A Flat Minima Perspective on Understanding Augmentations and Model Robustness","created":"2025-05-30","cited_by_count":1,"area":null},{"id":"2506.07159","title":"pFedSOP : Accelerating Training Of Personalized Federated Learning Using Second-Order Optimization","created":"2025-06-08","cited_by_count":1,"area":null},{"id":"2601.18420","title":"Gradient Regularized Natural Gradients","created":"2026-01-26","cited_by_count":1,"area":null},{"id":"2607.03871","title":"A Gradient Flow Perspective on Minimum MMD Estimation","created":"2026-07-04","cited_by_count":1,"area":null},{"id":"2109.10100","title":"A Novel Structured Natural Gradient Descent for Deep Learning","created":"2021-09-21","cited_by_count":0,"area":null},{"id":"2210.05268","title":"Component-Wise Natural Gradient Descent -- An Efficient Neural Network Optimization","created":"2022-10-11","cited_by_count":0,"area":null},{"id":"2302.13087","title":"Gauss-Newton Temporal Difference Learning with Nonlinear Function Approximation","created":"2023-02-25","cited_by_count":0,"area":null},{"id":"2302.13536","title":"Natural Gradient Hybrid Variational Inference with Application to Deep Mixed Models","created":"2023-02-27","cited_by_count":0,"area":null},{"id":"2305.11017","title":"Deep Metric Tensor Regularized Policy Gradient","created":"2023-05-18","cited_by_count":0,"area":null},{"id":"2310.14348","title":"DePAint: A Decentralized Safe Multi-Agent Reinforcement Learning Algorithm considering Peak and Average Constraints","created":"2023-10-22","cited_by_count":0,"area":null},{"id":"2403.03473","title":"Inverse-Free Fast Natural Gradient Descent Method for Deep Learning","created":"2024-03-06","cited_by_count":0,"area":null},{"id":"2504.19259","title":"Convergence Properties of Natural Gradient Descent for Minimizing KL Divergence","created":"2025-04-27","cited_by_count":0,"area":null},{"id":"2506.08428","title":"Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings","created":"2025-06-10","cited_by_count":0,"area":null},{"id":"2510.18812","title":"A Unified Perspective on Optimization in Machine Learning and Neuroscience: From Gradient Descent to Neural Adaptation","created":"2025-10-21","cited_by_count":0,"area":null},{"id":"2602.20078","title":"Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning","created":"2026-02-23","cited_by_count":0,"area":null},{"id":"2605.08352","title":"Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit","created":"2026-05-08","cited_by_count":0,"area":null},{"id":"2605.11911","title":"Understanding Sample Efficiency in Predictive Coding","created":"2026-05-12","cited_by_count":0,"area":null},{"id":"2605.18180","title":"Canonical Regularisation of Wide Feature-Learning Neural Networks","created":"2026-05-18","cited_by_count":0,"area":null}],"next_cursor":null}