mlsanityData to 7 Oct 2026

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models

arXiv 2403.13372 ↗First posted 20 Mar 2024cs.CL cs.AIcited by 1,442

Not on the map. Why?

Cited by 415 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Efficient fine-tuning is vital for adapting large language models (LLMs) to downstream tasks. However, it requires non-trivial efforts to implement these methods on different models. We present LlamaFactory, a unified framework that integrates a suite of cutting-edge efficient training methods. It provides a solution for flexibly customizing the fine-tuning of 100+ LLMs without the need for coding through the built-in web UI LlamaBoard. We empirically validate the efficiency and effectiveness of our framework on language modeling and text generation tasks. It has been released at https://github.com/hiyouga/LLaMA-Factory and received over 25,000 stars and 3,000 forks.

Key facts. Cited by 1,442 papers in the map (data through 7 Oct 2026). Builds on 64 of its 95 references that are in the map. First posted on arXiv 20 Mar 2024.

Builds on

64 of 95 references.

References · 64

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
38 more
  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  7. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  9. 2017Mixed Precision Trainingcited by 1,135
  10. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  11. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  12. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  13. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  14. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  15. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  16. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  17. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  18. 2024Yi: Open Foundation Models by 01.AIcited by 706
  19. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  20. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  21. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  22. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  23. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  24. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  25. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  26. 2023The Falcon Series of Open Language Modelscited by 526
  27. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  28. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 447
  29. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  30. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  31. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  32. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  33. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 365
  34. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  35. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  36. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  37. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
  38. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 231

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 174; 2025: 718; 2026: 550.

1,442 papers.

Sort:

By area:

  1. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  2. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  3. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  4. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 262
  5. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMscited by 246
  6. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  7. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 171
  8. 2025Agentic Reinforced Policy Optimizationcited by 166
  9. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  10. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 158
  11. 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
  12. 2024MAmmoTH2: Scaling Instructions from the Webcited by 143
38 more
  1. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 140
  2. 2025Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correctioncited by 140
  3. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilitiescited by 133
  4. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  5. 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 129
  6. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 129
  7. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  8. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 120
  9. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 119
  10. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  11. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 118
  12. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 115
  13. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 114
  14. 2025Small Models Struggle to Learn from Strong Reasonerscited by 109
  15. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
  16. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 101
  17. 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questionscited by 95
  18. 2025Reinforcement Learning for Self-Improving Agent with Skill Librarycited by 91
  19. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
  20. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 89
  21. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 88
  22. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 80
  23. 2025Agent Learning via Early Experiencecited by 79
  24. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 75
  25. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  26. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 69
  27. 2025Rank1: Test-Time Compute for Reranking in Information Retrievalcited by 68
  28. 2025VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learningcited by 68
  29. 2025Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoningcited by 67
  30. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 67
  31. 2024Agent Planning with World Knowledge Modelcited by 66
  32. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  33. 2025Think Only When You Need with Large Hybrid-Reasoning Modelscited by 64
  34. 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 63
  35. 2025EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modelingcited by 61
  36. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 59
  37. 2024PAD: Personalized Alignment of LLMs at Decoding-Timecited by 58
  38. 2024MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Routercited by 58

50 most cited of 1,442 All 1,442 (JSON)