{"list":"other","total":60,"rows":[{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380},{"id":"2110.14168","title":"Training Verifiers to Solve Math Word Problems","created":"2021-10-27","cited_by_count":10109},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1910.13461","title":"BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension","created":"2019-10-29","cited_by_count":5893},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"2108.07732","title":"Program Synthesis with Large Language Models","created":"2021-08-16","cited_by_count":3637},{"id":"1502.01852","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","created":"2015-02-06","cited_by_count":3611},{"id":"1802.05957","title":"Spectral Normalization for Generative Adversarial Networks","created":"2018-02-16","cited_by_count":1636},{"id":"1808.08745","title":"Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization","created":"2018-08-27","cited_by_count":1317},{"id":"1706.08498","title":"Spectrally-normalized margin bounds for neural networks","created":"2017-06-26","cited_by_count":686},{"id":"1711.00165","title":"Deep Neural Networks as Gaussian Processes","created":"2017-11-01","cited_by_count":622},{"id":"1706.02515","title":"Self-Normalizing Neural Networks","created":"2017-06-08","cited_by_count":598},{"id":"2102.00554","title":"Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks","created":"2021-01-31","cited_by_count":527},{"id":"2403.03507","title":"GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection","created":"2024-03-06","cited_by_count":463},{"id":"2502.16982","title":"Muon is Scalable for LLM Training","created":"2025-02-24","cited_by_count":428},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1705.09280","title":"Implicit Regularization in Matrix Factorization","created":"2017-05-25","cited_by_count":322},{"id":"1707.09564","title":"A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks","created":"2017-07-29","cited_by_count":314},{"id":"1704.08847","title":"Parseval Networks: Improving Robustness to Adversarial Examples","created":"2017-04-28","cited_by_count":293},{"id":"2306.07280","title":"Controlling Text-to-Image Diffusion by Orthogonal Finetuning","created":"2023-06-12","cited_by_count":208},{"id":"2307.05695","title":"ReLoRA: High-Rank Training Through Low-Rank Updates","created":"2023-07-11","cited_by_count":204},{"id":"2409.11321","title":"SOAP: Improving and Stabilizing Shampoo using Adam","created":"2024-09-17","cited_by_count":192},{"id":"1705.10941","title":"Spectral Norm Regularization for Improving the Generalizability of Deep Learning","created":"2017-05-31","cited_by_count":182},{"id":"1901.09021","title":"Complexity of Linear Regions in Deep Networks","created":"2019-01-25","cited_by_count":167},{"id":"2303.04129","title":"Foundation Models for Decision Making: Problems, Methods, and Opportunities","created":"2023-03-07","cited_by_count":164},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137},{"id":"2110.15343","title":"Scatterbrain: Unifying Sparse and Low-rank Attention Approximation","created":"2021-10-28","cited_by_count":130},{"id":"1712.07628","title":"Improving Generalization Performance by Switching from Adam to SGD","created":"2017-12-20","cited_by_count":125},{"id":"2311.06243","title":"Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization","created":"2023-11-10","cited_by_count":114},{"id":"2204.00595","title":"Monarch: Expressive Structured Matrices for Efficient and Accurate Training","created":"2022-04-01","cited_by_count":106},{"id":"1709.06079","title":"Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks","created":"2017-09-16","cited_by_count":92},{"id":"2505.02222","title":"Practical Efficiency of Muon for Pretraining","created":"2025-05-04","cited_by_count":80},{"id":"1903.05895","title":"Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations","created":"2019-03-14","cited_by_count":76},{"id":"1805.09298","title":"Learning towards Minimum Hyperspherical Energy","created":"2018-05-23","cited_by_count":75},{"id":"2410.01131","title":"nGPT: Normalized Transformer with Representation Learning on the Hypersphere","created":"2024-10-01","cited_by_count":70},{"id":"1711.03189","title":"Deep Hyperspherical Learning","created":"2017-11-08","cited_by_count":55},{"id":"1804.08071","title":"Decoupled Networks","created":"2018-04-22","cited_by_count":45},{"id":"2406.02214","title":"SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining","created":"2024-06-04","cited_by_count":45},{"id":"2004.04690","title":"Orthogonal Over-Parameterized Training","created":"2020-04-09","cited_by_count":44},{"id":"2107.01562","title":"Random Neural Networks in the Infinite Width Limit as Gaussian Processes","created":"2021-07-04","cited_by_count":44},{"id":"2012.07969","title":"A case for new neural network smoothness constraints","created":"2020-12-14","cited_by_count":43},{"id":"2410.01623","title":"Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?","created":"2024-10-02","cited_by_count":43},{"id":"2408.12857","title":"Memory-Efficient LLM Training with Online Subspace Descent","created":"2024-08-23","cited_by_count":39},{"id":"2407.08296","title":"Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients","created":"2024-07-11","cited_by_count":38},{"id":"2502.15280","title":"Hyperspherical Normalization for Scalable Deep Reinforcement Learning","created":"2025-02-21","cited_by_count":38},{"id":"2103.01649","title":"Learning with Hyperspherical Uniformity","created":"2021-03-02","cited_by_count":35},{"id":"2203.06345","title":"The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy","created":"2022-03-12","cited_by_count":35},{"id":"1611.03131","title":"Diverse Neural Network Learns True Target Functions","created":"2016-11-09","cited_by_count":33},{"id":"2303.10464","title":"SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models","created":"2023-03-18","cited_by_count":28},{"id":"2501.06842","title":"SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training","created":"2025-01-12","cited_by_count":28},{"id":"2502.10940","title":"CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation","created":"2025-02-16","cited_by_count":28},{"id":"2412.13148","title":"SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training","created":"2024-12-17","cited_by_count":20},{"id":"2405.17991","title":"VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections","created":"2024-05-28","cited_by_count":18},{"id":"2402.16828","title":"Training Neural Networks from Scratch with Parallel Low-Rank Adapters","created":"2024-02-26","cited_by_count":17},{"id":"2504.20437","title":"GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection","created":"2025-04-29","cited_by_count":14},{"id":"2412.19616","title":"Gradient Weight-normalized Low-rank Projection for Efficient LLM Training","created":"2024-12-27","cited_by_count":11},{"id":"2412.06289","title":"S$^{2}$FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity","created":"2024-12-09","cited_by_count":10},{"id":"2412.19820","title":"GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection","created":"2024-12-15","cited_by_count":8}],"next_cursor":null}