{"list":"other","total":55,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1710.10903","title":"Graph Attention Networks","created":"2017-10-30","cited_by_count":8923},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"2201.03545","title":"A ConvNet for the 2020s","created":"2022-01-10","cited_by_count":3579},{"id":"2304.01373","title":"Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling","created":"2023-04-03","cited_by_count":1786},{"id":"1910.07467","title":"Root Mean Square Layer Normalization","created":"2019-10-16","cited_by_count":1480},{"id":"2406.17557","title":"The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale","created":"2024-06-25","cited_by_count":1160},{"id":"2606.19348","title":"DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence","created":"2026-04-26","cited_by_count":1040},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"2404.06395","title":"MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies","created":"2024-04-09","cited_by_count":723},{"id":"1612.08083","title":"Language Modeling with Gated Convolutional Networks","created":"2016-12-23","cited_by_count":702},{"id":"1802.04434","title":"signSGD: Compressed Optimisation for Non-Convex Problems","created":"2018-02-13","cited_by_count":571},{"id":"2412.06464","title":"Gated Delta Networks: Improving Mamba2 with Delta Rule","created":"2024-12-09","cited_by_count":532},{"id":"2305.07759","title":"TinyStories: How Small Can Language Models Be and Still Speak Coherent English?","created":"2023-05-12","cited_by_count":449},{"id":"2106.14881","title":"Early Convolutions Help Transformers See Better","created":"2021-06-28","cited_by_count":396},{"id":"2403.13298","title":"Rotary Position Embedding for Vision Transformer","created":"2024-03-20","cited_by_count":236},{"id":"1905.12558","title":"Limitations of the Empirical Fisher Approximation for Natural Gradient Descent","created":"2019-05-29","cited_by_count":217},{"id":"2502.07529","title":"Training Deep Learning Models with Norm-Constrained LMOs","created":"2025-02-11","cited_by_count":192},{"id":"2309.14322","title":"Small-scale proxies for large-scale Transformer training instabilities","created":"2023-09-25","cited_by_count":183},{"id":"2109.08668","title":"Primer: Searching for Efficient Transformers for Language Modeling","created":"2021-09-17","cited_by_count":164},{"id":"2305.17589","title":"Graph Inductive Biases in Transformers without Message Passing","created":"2023-05-27","cited_by_count":153},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141},{"id":"1705.07774","title":"Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients","created":"2017-05-22","cited_by_count":139},{"id":"2002.10542","title":"Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence","created":"2020-02-24","cited_by_count":137},{"id":"2010.05627","title":"Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning","created":"2020-10-12","cited_by_count":126},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124},{"id":"2007.01547","title":"Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers","created":"2020-07-03","cited_by_count":119},{"id":"1907.04164","title":"Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model","created":"2019-07-09","cited_by_count":117},{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114},{"id":"2505.16932","title":"The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm","created":"2025-05-22","cited_by_count":110},{"id":"2205.10287","title":"On the SDEs and Scaling Rules for Adaptive Gradient Algorithms","created":"2022-05-20","cited_by_count":104},{"id":"2310.01082","title":"Linear attention is (maybe) all you need (to understand transformer optimization)","created":"2023-10-02","cited_by_count":98},{"id":"2503.12645","title":"Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization","created":"2025-03-16","cited_by_count":92},{"id":"2106.14343","title":"High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails","created":"2021-06-28","cited_by_count":80},{"id":"2402.19449","title":"Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models","created":"2024-02-29","cited_by_count":80},{"id":"2505.02222","title":"Practical Efficiency of Muon for Pretraining","created":"2025-05-04","cited_by_count":80},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73},{"id":"2410.01131","title":"nGPT: Normalized Transformer with Representation Learning on the Hypersphere","created":"2024-10-01","cited_by_count":70},{"id":"1705.05933","title":"Sub-sampled Cubic Regularization for Non-convex Optimization","created":"2017-05-16","cited_by_count":63},{"id":"2306.07179","title":"Benchmarking Neural Network Training Algorithms","created":"2023-06-12","cited_by_count":52},{"id":"2410.13849","title":"From Gradient Clipping to Normalization for Heavy Tailed SGD","created":"2024-10-17","cited_by_count":49},{"id":"2507.07101","title":"Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful","created":"2025-07-09","cited_by_count":41},{"id":"2505.21829","title":"In Search of Adam's Secret Sauce","created":"2025-05-27","cited_by_count":40},{"id":"2502.15015","title":"Accelerating Neural Network Training: An Analysis of the AlgoPerf Competition","created":"2025-02-20","cited_by_count":32},{"id":"2405.21064","title":"Recurrent neural networks: vanishing and exploding gradients are not the end of the story","created":"2024-05-31","cited_by_count":25},{"id":"2505.19227","title":"Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law","created":"2025-05-25","cited_by_count":24},{"id":"2211.09359","title":"How to Fine-Tune Vision Models with SGD","created":"2022-11-17","cited_by_count":23},{"id":"2411.15958","title":"Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise","created":"2024-11-24","cited_by_count":23},{"id":"2508.04860","title":"Can SGD Handle Heavy-Tailed Noise?","created":"2025-08-06","cited_by_count":17},{"id":"2506.12543","title":"Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling","created":"2025-06-14","cited_by_count":15},{"id":"2603.15958","title":"Deriving Hyperparameter Scaling Laws via Modern Optimization Theory","created":"2026-03-16","cited_by_count":10},{"id":"2510.19376","title":"Optimization Benchmark for Diffusion Models on Dynamical Systems","created":"2025-10-22","cited_by_count":7},{"id":"2603.21191","title":"On the Role of Batch Size in Stochastic Conditional Gradient Methods","created":"2026-03-22","cited_by_count":7}],"next_cursor":null}