mlsanityData to 14 Oct 2026

LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models

arXiv 2308.10252 ↗First posted 20 Aug 2023cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

With the burgeoning development in the realm of large language models (LLMs), the demand for efficient incremental training tailored to specific industries and domains continues to increase. Currently, the predominantly employed frameworks lack modular design, it often takes a lot of coding work to kickstart the training of LLM. To address this, we present "LMTuner", a highly usable, integrable, and scalable system for training LLMs expeditiously and with minimal user-input. LMTuner comprises three main modules - the Interaction, Training, and Inference Modules. We advocate that LMTuner's usability and integrality alleviate the complexities in training large language models. Remarkably, even a novice user could commence training large language models within five minutes. Furthermore, it integrates DeepSpeed frameworks and supports Efficient Fine-Tuning methodologies like Low Rank Adaptation (LoRA), Quantized LoRA (QLoRA), etc., enabling the training of language models scaling from 300M to a whopping 130B parameters using a single server. The LMTuner's homepage (https://wengsyx.github.io/LMTuner/)and screencast video (https://youtu.be/nsXmWOmN3rE) are now publicly available.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 46 of its 61 references that are in the map. First posted on arXiv 20 Aug 2023.

Builds on

46 of 61 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  10. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
34 more
  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  9. 2023LIMA: Less Is More for Alignmentcited by 1,100
  10. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  11. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  12. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  13. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  14. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  15. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  16. 2023Symbolic Discovery of Optimization Algorithmscited by 434
  17. 2023Large Language Models as Tool Makerscited by 303
  18. 2020MedDialog: Two Large-scale Medical Dialogue Datasetscited by 161
  19. 2023Full Parameter Fine-tuning for Large Language Models with Limited Resourcescited by 134
  20. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  21. 2022A Length-Extrapolatable Transformercited by 127
  22. 2023Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecturecited by 122
  23. 2023Training Transformers with 4-bit Integerscited by 60
  24. 2023ZeRO++: Extremely Efficient Collective Communication for Giant Model Trainingcited by 52
  25. 2023Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Studycited by 43
  26. 2022Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysiscited by 42
  27. 2023CAME: Confidence-guided Adaptive Memory Efficient Optimizationcited by 36
  28. 2023Interactive Natural Language Processingcited by 31
  29. 2022TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalitiescited by 19
  30. 2022LingYi: Medical Conversational Question Answering System based on Multi-modal Knowledge Graphscited by 15
  31. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
  32. 2023OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Modelscited by 6
  33. 2023Large Language Models Need Holistically Thought in Medical Conversational QAcited by 5
  34. 2023Towards Graph-hop Retrieval and Reasoning in Complex Question Answering over Textual Databasecited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024A Fine-tuning Enhanced RAG System with Quantized Influence Measure as AI Judgecited by 9
  2. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Modelscited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2025Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
  2. 2024Low-Rank Quantization-Aware Training for LLMs
  3. 2024L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models
  4. 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report
  5. 2025L-MoE: End-to-End Training of a Lightweight Mixture of Low-Rank Adaptation Experts
  6. 2025Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
  7. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models
  8. 2025LoRA Is Slower Than You Think