{"list":"other","total":64,"rows":[{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"2203.02155","title":"Training language models to follow instructions with human feedback","created":"2022-03-04","cited_by_count":15971},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2404.14219","title":"Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone","created":"2024-04-22","cited_by_count":1917},{"id":"2010.11934","title":"mT5: A massively multilingual pre-trained text-to-text transformer","created":"2020-10-22","cited_by_count":1818},{"id":"1804.04235","title":"Adafactor: Adaptive Learning Rates with Sublinear Memory Cost","created":"2018-04-11","cited_by_count":905},{"id":"2104.04473","title":"Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM","created":"2021-04-09","cited_by_count":632},{"id":"2403.03507","title":"GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection","created":"2024-03-06","cited_by_count":463},{"id":"2110.02861","title":"8-bit Optimizers via Block-wise Quantization","created":"2021-10-06","cited_by_count":403},{"id":"1812.04754","title":"Gradient Descent Happens in a Tiny Subspace","created":"2018-12-12","cited_by_count":217},{"id":"2307.05695","title":"ReLoRA: High-Rank Training Through Low-Rank Updates","created":"2023-07-11","cited_by_count":204},{"id":"2310.04948","title":"TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting","created":"2023-10-08","cited_by_count":174},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137},{"id":"2402.03293","title":"Flora: Low-Rank Adapters Are Secretly Gradient Compressors","created":"2024-02-05","cited_by_count":108},{"id":"2403.17919","title":"LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning","created":"2024-03-26","cited_by_count":89},{"id":"2010.11364","title":"Sample Efficient Reinforcement Learning with REINFORCE","created":"2020-10-22","cited_by_count":86},{"id":"2401.04151","title":"Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning","created":"2024-01-08","cited_by_count":82},{"id":"2309.02411","title":"Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices","created":"2023-09-05","cited_by_count":74},{"id":"2309.01507","title":"Memory Efficient Optimizers with 4-bit States","created":"2023-09-04","cited_by_count":54},{"id":"2406.02214","title":"SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining","created":"2024-06-04","cited_by_count":45},{"id":"2410.01623","title":"Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?","created":"2024-10-02","cited_by_count":43},{"id":"2408.12857","title":"Memory-Efficient LLM Training with Online Subspace Descent","created":"2024-08-23","cited_by_count":39},{"id":"2407.08296","title":"Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients","created":"2024-07-11","cited_by_count":38},{"id":"2307.02047","title":"CAME: Confidence-guided Adaptive Memory Efficient Optimization","created":"2023-07-05","cited_by_count":36},{"id":"2405.16002","title":"Does SGD really happen in tiny subspaces?","created":"2024-05-25","cited_by_count":36},{"id":"2305.00660","title":"An Iterative Algorithm for Rescaled Hyperbolic Functions Regression","created":"2023-05-01","cited_by_count":35},{"id":"2312.17482","title":"MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining","created":"2023-12-29","cited_by_count":34},{"id":"2410.11289","title":"Subspace Optimization for Large Language Models with Convergence Guarantees","created":"2024-10-15","cited_by_count":34},{"id":"2307.02419","title":"In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick","created":"2023-07-05","cited_by_count":32},{"id":"2405.05219","title":"Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers","created":"2024-05-08","cited_by_count":32},{"id":"2302.11068","title":"Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time","created":"2023-02-21","cited_by_count":31},{"id":"2306.02295","title":"A Mathematical Abstraction for Balancing the Trade-off Between Creativity and Reality in Large Language Models","created":"2023-06-04","cited_by_count":29},{"id":"2403.11886","title":"QueryAgent: A Reliable and Efficient Reasoning Framework with Environmental Feedback-based Self-Correction","created":"2024-03-18","cited_by_count":27},{"id":"2308.10502","title":"GradientCoin: A Peer-to-Peer Decentralized Large Language Models","created":"2023-08-21","cited_by_count":23},{"id":"2407.05952","title":"H-STAR: LLM-driven Hybrid SQL-Text Adaptive Reasoning on Tables","created":"2024-06-29","cited_by_count":21},{"id":"2412.06148","title":"The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity","created":"2024-12-09","cited_by_count":21},{"id":"2308.14304","title":"Solving Attention Kernel Regression Problem via Pre-conditioner","created":"2023-08-28","cited_by_count":20},{"id":"2411.08516","title":"Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understanding","created":"2024-11-13","cited_by_count":20},{"id":"2402.16141","title":"PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization","created":"2024-02-25","cited_by_count":19},{"id":"2404.18466","title":"HFT: Half Fine-Tuning for Large Language Models","created":"2024-04-29","cited_by_count":18},{"id":"2406.17660","title":"Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients","created":"2024-06-25","cited_by_count":18},{"id":"2412.17316","title":"RoPE Attention Can Be Trained in Almost Linear Time","created":"2024-12-23","cited_by_count":13},{"id":"2309.13482","title":"A Unified Scheme of ResNet and Softmax","created":"2023-09-23","cited_by_count":12},{"id":"1904.01145","title":"Stochastic Subspace Descent","created":"2019-04-01","cited_by_count":10},{"id":"2306.04169","title":"Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation","created":"2023-06-07","cited_by_count":9},{"id":"2311.15390","title":"Local Convergence of Approximate Newton Method for Two Layer Nonlinear Regression","created":"2023-11-26","cited_by_count":9},{"id":"2403.14958","title":"Adapprox: Adaptive Approximation in Adam Optimization via Randomized Low-Rank Matrices","created":"2024-03-22","cited_by_count":8},{"id":"2404.10851","title":"Sample Complexity of the Linear Quadratic Regulator: A Reinforcement Learning Lens","created":"2024-04-16","cited_by_count":8},{"id":"2405.18380","title":"Outlier-weighed Layerwise Sampling for LLM Fine-tuning","created":"2024-05-28","cited_by_count":8},{"id":"2306.10694","title":"On the Model-Misspecification in Reinforcement Learning","created":"2023-06-19","cited_by_count":7},{"id":"2310.20051","title":"The Expressibility of Polynomial based Attention Scheme","created":"2023-10-30","cited_by_count":7},{"id":"2411.00401","title":"Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayes Theory","created":"2024-11-01","cited_by_count":7},{"id":"2208.03915","title":"Dynamic Maintenance of Kernel Density Estimation Data Structure: From Practice to Theory","created":"2022-08-08","cited_by_count":6},{"id":"2306.09554","title":"Low-Switching Policy Gradient with Exploration via Online Sensitivity Sampling","created":"2023-06-15","cited_by_count":6},{"id":"2309.10129","title":"Adaptive Liquidity Provision in Uniswap V3 with Deep Reinforcement Learning","created":"2023-09-18","cited_by_count":6},{"id":"2402.12711","title":"Uniform Last-Iterate Guarantee for Bandits and Reinforcement Learning","created":"2024-02-20","cited_by_count":5},{"id":"2412.13488","title":"Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models","created":"2024-12-18","cited_by_count":3},{"id":"2405.06003","title":"Binary Hypothesis Testing for Softmax Models and Leverage Score Models","created":"2024-05-09","cited_by_count":1}],"next_cursor":null}