{"total":141,"rows":[{"id":"2107.03342","title":"A Survey of Uncertainty in Deep Neural Networks","created":"2021-07-07","cited_by_count":680,"area":null},{"id":"2111.09832","title":"Merging Models with Fisher-Weighted Averaging","created":"2021-11-18","cited_by_count":621,"area":null},{"id":"2208.11580","title":"Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning","created":"2022-08-24","cited_by_count":322,"area":null},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265,"area":null},{"id":"2405.13954","title":"What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions","created":"2024-05-22","cited_by_count":94,"area":null},{"id":"2309.06497","title":"A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale","created":"2023-09-12","cited_by_count":77,"area":null},{"id":"2210.05177","title":"Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach","created":"2022-10-11","cited_by_count":70,"area":null},{"id":"2107.03356","title":"M-FAC: Efficient Matrix-Free Approximations of Second-Order Information","created":"2021-07-07","cited_by_count":54,"area":null},{"id":"2110.01765","title":"Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping","created":"2021-10-05","cited_by_count":52,"area":null},{"id":"2105.02716","title":"Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks","created":"2021-05-06","cited_by_count":50,"area":null},{"id":"2311.00636","title":"Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures","created":"2023-11-01","cited_by_count":49,"area":null},{"id":"2202.10638","title":"Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations","created":"2022-02-22","cited_by_count":42,"area":null},{"id":"2502.04664","title":"Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data","created":"2025-02-07","cited_by_count":41,"area":null},{"id":"2312.04339","title":"Merging by Matching Models in Task Parameter Subspaces","created":"2023-12-07","cited_by_count":36,"area":null},{"id":"2405.12186","title":"Training Data Attribution via Approximate Unrolled Differentiation","created":"2024-05-20","cited_by_count":34,"area":null},{"id":"2410.13850","title":"Influence Functions for Scalable Data Attribution in Diffusion Models","created":"2024-10-17","cited_by_count":33,"area":"diffusion"},{"id":"2201.12250","title":"Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization","created":"2022-01-28","cited_by_count":30,"area":null},{"id":"2502.19002","title":"The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training","created":"2025-02-26","cited_by_count":30,"area":null},{"id":"2106.02925","title":"Tensor Normal Training for Deep Learning Models","created":"2021-06-05","cited_by_count":24,"area":null},{"id":"2310.06131","title":"Learning Layer-wise Equivariances Automatically using Gradients","created":"2023-10-09","cited_by_count":23,"area":null},{"id":"2506.03595","title":"Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner","created":"2025-06-04","cited_by_count":23,"area":null},{"id":"2310.00339","title":"FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation","created":"2023-09-30","cited_by_count":22,"area":null},{"id":"2204.02738","title":"Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network","created":"2022-04-06","cited_by_count":21,"area":null},{"id":"2405.15603","title":"Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks","created":"2024-05-24","cited_by_count":21,"area":null},{"id":"2512.05620","title":"Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales","created":"2025-12-05","cited_by_count":21,"area":null},{"id":"2107.01739","title":"KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks","created":"2021-07-04","cited_by_count":20,"area":null},{"id":"2106.02624","title":"ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure","created":"2021-06-04","cited_by_count":16,"area":null},{"id":"2110.14109","title":"Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums","created":"2021-10-27","cited_by_count":15,"area":null},{"id":"2312.12226","title":"On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width","created":"2023-12-19","cited_by_count":15,"area":null},{"id":"2403.12329","title":"FedFisher: Leveraging Fisher Information for One-Shot Federated Learning","created":"2024-03-19","cited_by_count":15,"area":null},{"id":"2405.16397","title":"AdaFisher: Adaptive Second Order Optimization via Fisher Information","created":"2024-05-26","cited_by_count":14,"area":null},{"id":"2405.20763","title":"Improving Generalization and Convergence by Enhancing Implicit Regularization","created":"2024-05-31","cited_by_count":14,"area":null},{"id":"2405.10927","title":"Using Degeneracy in the Loss Landscape for Mechanistic Interpretability","created":"2024-05-17","cited_by_count":13,"area":"mechinterp"},{"id":"2406.00104","title":"Scalable Bayesian Learning with posteriors","created":"2024-05-31","cited_by_count":13,"area":null},{"id":"2410.02117","title":"Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices","created":"2024-10-03","cited_by_count":13,"area":"moe"},{"id":"2109.14158","title":"Second-Order Neural ODE Optimizer","created":"2021-09-29","cited_by_count":11,"area":null},{"id":"2206.15273","title":"Invariance Properties of the Natural Gradient in Overparametrised Systems","created":"2022-06-30","cited_by_count":11,"area":null},{"id":"2208.08112","title":"DLCFT: Deep Linear Continual Fine-Tuning for General Incremental Learning","created":"2022-08-17","cited_by_count":11,"area":null},{"id":"2111.00856","title":"Large-Scale Deep Learning Optimizations: A Comprehensive Survey","created":"2021-11-01","cited_by_count":10,"area":null},{"id":"2302.09738","title":"Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning","created":"2023-02-20","cited_by_count":10,"area":null},{"id":"2507.14740","title":"Better Training Data Attribution via Better Inverse Hessian-Vector Products","created":"2025-07-19","cited_by_count":10,"area":null},{"id":"2602.17835","title":"Influence-Preserving Proxies for Gradient-Based Data Selection in LLM Fine-tuning","created":"2026-02-19","cited_by_count":10,"area":null},{"id":"2201.10285","title":"Efficient Approximations of the Fisher Matrix in Neural Networks using Kronecker Product Singular Value Decomposition","created":"2022-01-25","cited_by_count":9,"area":null},{"id":"2203.00089","title":"Amortized Proximal Optimization","created":"2022-02-28","cited_by_count":9,"area":null},{"id":"2312.05705","title":"Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC","created":"2023-12-09","cited_by_count":9,"area":null},{"id":"2501.19183","title":"Position: Curvature Matrices Should Be Democratized via Linear Operators","created":"2025-01-31","cited_by_count":9,"area":null},{"id":"2102.07725","title":"Neural Network Compression for Noisy Storage Devices","created":"2021-02-15","cited_by_count":8,"area":null},{"id":"2202.04124","title":"A Mini-Block Fisher Method for Deep Neural Networks","created":"2022-02-08","cited_by_count":8,"area":null},{"id":"2303.05473","title":"Natural Gradient Methods: Perspectives, Efficient-Scalable Approximations, and Analysis","created":"2023-03-06","cited_by_count":8,"area":null},{"id":"2307.10438","title":"Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search","created":"2023-07-19","cited_by_count":8,"area":null},{"id":"2505.17974","title":"Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models","created":"2025-05-23","cited_by_count":8,"area":null},{"id":"2602.09314","title":"Clarifying Shampoo: Adapting Spectral Descent to Stochasticity and the Parameter Trajectory","created":"2026-02-10","cited_by_count":8,"area":null},{"id":"2106.03947","title":"TENGraD: Time-Efficient Natural Gradient Descent with Exact Fisher-Block Inversion","created":"2021-06-07","cited_by_count":7,"area":null},{"id":"2206.15143","title":"Scalable K-FAC Training for Deep Neural Networks with Distributed Preconditioning","created":"2022-06-30","cited_by_count":7,"area":null},{"id":"2302.03519","title":"Efficient Parametric Approximations of Neural Network Function Space Distance","created":"2023-02-07","cited_by_count":7,"area":null},{"id":"2410.19921","title":"Method for noise-induced regularization in quantum neural networks","created":"2024-10-25","cited_by_count":7,"area":null},{"id":"2412.10782","title":"ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning","created":"2024-12-14","cited_by_count":7,"area":null},{"id":"2507.01649","title":"GradMetaNet: An Equivariant Architecture for Learning on Gradients","created":"2025-07-02","cited_by_count":7,"area":null},{"id":"2102.06737","title":"Kronecker-factored Quasi-Newton Methods for Deep Learning","created":"2021-02-12","cited_by_count":6,"area":null},{"id":"2302.02904","title":"Rethinking Gauss-Newton for learning over-parameterized models","created":"2023-02-06","cited_by_count":6,"area":null},{"id":"2305.00604","title":"ISAAC Newton: Input-based Approximate Curvature for Newton's Method","created":"2023-05-01","cited_by_count":6,"area":null},{"id":"2306.06098","title":"Error Feedback Can Accurately Compress Preconditioners","created":"2023-06-09","cited_by_count":6,"area":null},{"id":"2509.22335","title":"Spectral Collapse Drives Loss of Plasticity in Deep Continual Learning","created":"2025-09-26","cited_by_count":6,"area":null},{"id":"2107.06533","title":"Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks","created":"2021-07-14","cited_by_count":5,"area":null},{"id":"2111.11542","title":"Depth Without the Magic: Inductive Bias of Natural Gradient Descent","created":"2021-11-22","cited_by_count":5,"area":null},{"id":"2207.07287","title":"Riemannian Natural Gradient Methods","created":"2022-07-15","cited_by_count":5,"area":null},{"id":"2301.05785","title":"Efficient Activation Function Optimization through Surrogate Modeling","created":"2023-01-13","cited_by_count":5,"area":null},{"id":"2310.05884","title":"A Meta-Learning Perspective on Transformers for Causal Language Modeling","created":"2023-10-09","cited_by_count":5,"area":null},{"id":"2410.09568","title":"Second-Order Min-Max Optimization with Lazy Hessians","created":"2024-10-12","cited_by_count":5,"area":null},{"id":"2602.19926","title":"Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models","created":"2026-02-23","cited_by_count":5,"area":"lr"},{"id":"2110.08577","title":"Nys-Newton: Nystr\\\"om-Approximated Curvature for Stochastic Optimization","created":"2021-10-16","cited_by_count":4,"area":null},{"id":"2111.08577","title":"Neuron-based Pruning of Deep Neural Networks with Better Generalization using Kronecker Factored Curvature Approximation","created":"2021-11-16","cited_by_count":4,"area":null},{"id":"2303.04012","title":"Exploration via Epistemic Value Estimation","created":"2023-03-07","cited_by_count":4,"area":null},{"id":"2306.17504","title":"Systematic Investigation of Sparse Perturbed Sharpness-Aware Minimization Optimizer","created":"2023-06-30","cited_by_count":4,"area":null},{"id":"2307.07753","title":"Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks","created":"2023-07-15","cited_by_count":4,"area":null},{"id":"2403.00158","title":"Automated Efficient Estimation using Monte Carlo Efficient Influence Functions","created":"2024-02-29","cited_by_count":4,"area":null},{"id":"2410.08087","title":"Noether's razor: Learning Conserved Quantities","created":"2024-10-10","cited_by_count":4,"area":null},{"id":"2412.02328","title":"Efficient Model Compression Techniques with FishLeg","created":"2024-12-03","cited_by_count":4,"area":null},{"id":"2507.08091","title":"Low-rank Momentum Factorization for Memory Efficient Training","created":"2025-07-10","cited_by_count":4,"area":null},{"id":"2206.15397","title":"Randomized K-FACs: Speeding up K-FAC with Randomized Numerical Linear Algebra","created":"2022-06-30","cited_by_count":3,"area":null},{"id":"2503.15367","title":"FedBEns: One-Shot Federated Learning based on Bayesian Ensemble","created":"2025-03-19","cited_by_count":3,"area":null},{"id":"2507.03117","title":"BLaST: High Performance Inference and Pretraining using BLock Sparse Transformers","created":"2025-07-03","cited_by_count":3,"area":null},{"id":"2602.17385","title":"Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature","created":"2026-02-19","cited_by_count":3,"area":null},{"id":"2606.19491","title":"Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale","created":"2026-06-17","cited_by_count":3,"area":null},{"id":"2101.00218","title":"An iterative K-FAC algorithm for Deep Learning","created":"2021-01-01","cited_by_count":2,"area":null},{"id":"2106.07903","title":"Robust Out-of-Distribution Detection on Deep Probabilistic Generative Models","created":"2021-06-15","cited_by_count":2,"area":null},{"id":"2107.04695","title":"L2M: Practical posterior Laplace approximation with optimization-driven second moment estimation","created":"2021-07-09","cited_by_count":2,"area":null},{"id":"2202.06232","title":"Generalized Tangent Kernel: A Unified Geometric Foundation for Natural Gradient and Standard Gradient","created":"2022-02-13","cited_by_count":2,"area":null},{"id":"2206.10936","title":"Information Geometry of Dropout Training","created":"2022-06-22","cited_by_count":2,"area":null},{"id":"2207.01847","title":"PoF: Post-Training of Feature Extractor for Improving Generalization","created":"2022-07-05","cited_by_count":2,"area":null},{"id":"2210.08494","title":"Brand New K-FACs: Speeding up K-FAC with Online Decomposition Updates","created":"2022-10-16","cited_by_count":2,"area":null},{"id":"2310.04649","title":"Uncovering Model Processing Strategies with Non-Negative Per-Example Fisher Factorization","created":"2023-10-07","cited_by_count":2,"area":null},{"id":"2410.14325","title":"Debiasing Mini-Batch Quadratics for Applications in Deep Learning","created":"2024-10-18","cited_by_count":2,"area":null},{"id":"2506.18588","title":"Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks","created":"2025-06-23","cited_by_count":2,"area":null},{"id":"2507.03545","title":"DOME: Improving Signal-to-Noise in Stochastic Gradient Descent via Sharp-Direction Subspace Filtering","created":"2025-07-04","cited_by_count":2,"area":null},{"id":"2601.12816","title":"Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning","created":"2026-01-19","cited_by_count":2,"area":null},{"id":"2601.20769","title":"Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidence","created":"2026-01-28","cited_by_count":2,"area":null},{"id":"2605.04712","title":"SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning","created":"2026-05-06","cited_by_count":2,"area":"moe"},{"id":"2605.08746","title":"The Global Empirical NTK: Self-Referential Bias and Dimensionality of Gradient Descent Learning","created":"2026-05-09","cited_by_count":2,"area":null},{"id":"2605.18528","title":"Tight Complexity of Muon under Heavy-Tailed Noise","created":"2026-05-18","cited_by_count":2,"area":null}],"next_cursor":"100"}