mlsanityData to 14 Oct 2026

Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models

arXiv 2508.17184 ↗First posted 24 Aug 2025cs.CLcited by 5

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Instruction tuning is a pivotal technique for aligning large language models (LLMs) with human intentions, safety constraints, and domain-specific requirements. This survey provides a comprehensive overview of the full pipeline, encompassing (i) data collection methodologies, (ii) full-parameter and parameter-efficient fine-tuning strategies, and (iii) evaluation protocols. We categorized data construction into three major paradigms: expert annotation, distillation from larger models, and self-improvement mechanisms, each offering distinct trade-offs between quality, scalability, and resource cost. Fine-tuning techniques range from conventional supervised training to lightweight approaches, such as low-rank adaptation (LoRA) and prefix tuning, with a focus on computational efficiency and model reusability. We further examine the challenges of evaluating faithfulness, utility, and safety across multilingual and multimodal scenarios, highlighting the emergence of domain-specific benchmarks in healthcare, legal, and financial applications. Finally, we discuss promising directions for automated data generation, adaptive optimization, and robust evaluation frameworks, arguing that a closer integration of data, algorithms, and human feedback is essential for advancing instruction-tuned LLMs. This survey aims to serve as a practical reference for researchers and practitioners seeking to design LLMs that are both effective and reliably aligned with human intentions.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 96 of its 109 references that are in the map. First posted on arXiv 24 Aug 2025.

Builds on

96 of 109 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search14 Oct
  2. 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities5 Sep
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  4. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 92

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  12. 2017Deep reinforcement learning from human preferencescited by 3,912
38 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  9. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  10. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  11. 2022Red Teaming Language Models with Language Modelscited by 1,026
  12. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  13. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  14. 2019Are Sixteen Heads Really Better than One?cited by 816
  15. 2023A Comprehensive Overview of Large Language Modelscited by 648
  16. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  17. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 460
  18. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  19. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  20. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
  21. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  22. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 333
  23. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  24. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  25. 2023A Survey on Model Compression for Large Language Modelscited by 269
  26. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  27. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 243
  28. 2023Inverse Scaling: When Bigger Isn't Bettercited by 207
  29. 2024Large Language Models for Education: A Survey and Outlookcited by 188
  30. 2024A Closer Look at the Limitations of Instruction Tuningcited by 77
  31. 2023Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMscited by 64
  32. 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunitiescited by 64
  33. 2023Maybe Only 0.5% Data is Needed: A Preliminary Exploration of Low Training Data Instruction Tuningcited by 58
  34. 2023A Bibliometric Review of Large Language Models Research from 2017 to 2023cited by 57
  35. 2023From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuningcited by 56
  36. 2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 48
  37. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  38. 2024Everybody Prune Now: Structured Pruning of LLMs with only Forward Passescited by 43

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Aligning Instruction Tuning with Pre-training
  2. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
  3. 2025MAIN: Mutual Alignment Is Necessary for instruction tuning
  4. 2025A Comprehensive Evaluation framework of Alignment Techniques for LLMs
  5. 2024CodecLM: Aligning Language Models with Tailored Synthetic Data
  6. 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
  7. 2024Improving the Robustness of Large Language Models via Consistency Alignment
  8. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning