{"total":19,"rows":[{"id":"2208.06677","title":"Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models","created":"2022-08-13","cited_by_count":174,"area":null},{"id":"2108.01624","title":"Large-Scale Differentially Private BERT","created":"2021-08-03","cited_by_count":118,"area":null},{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114,"area":null},{"id":"2106.04015","title":"Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning","created":"2021-06-07","cited_by_count":82,"area":null},{"id":"2106.07820","title":"On Large-Cohort Training for Federated Learning","created":"2021-06-15","cited_by_count":74,"area":null},{"id":"2105.02716","title":"Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks","created":"2021-05-06","cited_by_count":50,"area":null},{"id":"2110.04369","title":"A Loss Curvature Perspective on Training Instability in Deep Learning","created":"2021-10-08","cited_by_count":40,"area":null},{"id":"2401.14489","title":"The Case for Co-Designing Model Architectures with Hardware","created":"2024-01-25","cited_by_count":11,"area":null},{"id":"2111.00856","title":"Large-Scale Deep Learning Optimizations: A Comprehensive Survey","created":"2021-11-01","cited_by_count":10,"area":null},{"id":"2306.08590","title":"Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning","created":"2023-06-14","cited_by_count":10,"area":null},{"id":"2203.01400","title":"Adaptive Gradient Methods with Local Guarantees","created":"2022-03-02","cited_by_count":8,"area":null},{"id":"2309.13681","title":"Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR)","created":"2023-09-24","cited_by_count":3,"area":null},{"id":"2112.08250","title":"Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach","created":"2021-12-15","cited_by_count":2,"area":null},{"id":"2602.03001","title":"Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent","created":"2026-02-03","cited_by_count":2,"area":null},{"id":"2304.11692","title":"The Disharmony between BN and ReLU Causes Gradient Explosion, but is Offset by the Correlation between Activations","created":"2023-04-23","cited_by_count":1,"area":null},{"id":"2108.05839","title":"Logit Attenuating Weight Normalization","created":"2021-08-12","cited_by_count":0,"area":null},{"id":"2201.12469","title":"ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise","created":"2022-01-29","cited_by_count":0,"area":null},{"id":"2208.03941","title":"Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks","created":"2022-08-08","cited_by_count":0,"area":null},{"id":"2311.13755","title":"Transformer-based Named Entity Recognition in Construction Supply Chain Risk Management in Australia","created":"2023-11-23","cited_by_count":0,"area":null}],"next_cursor":null}