Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods
Abstract (arXiv)
Instruction tuning is a pivotal technique for aligning large language models (LLMs) with human intentions, safety constraints, and domain-specific requirements. This survey provides a comprehensive overview of the full pipeline, encompassing (i) data collection methodologies, (ii) full-parameter and parameter-efficient fine-tuning strategies, and (iii) evaluation protocols. We categorized data construction into three major paradigms: expert annotation, distillation from larger models, and self-improvement mechanisms, each offering distinct trade-offs between quality, scalability, and resource cost. Fine-tuning techniques range from conventional supervised training to lightweight approaches, such as low-rank adaptation (LoRA) and prefix tuning, with a focus on computational efficiency and model reusability. We further examine the challenges of evaluating faithfulness, utility, and safety across multilingual and multimodal scenarios, highlighting the emergence of domain-specific benchmarks in healthcare, legal, and financial applications. Finally, we discuss promising directions for automated data generation, adaptive optimization, and robust evaluation frameworks, arguing that a closer integration of data, algorithms, and human feedback is essential for advancing instruction-tuned LLMs. This survey aims to serve as a practical reference for researchers and practitioners seeking to design LLMs that are both effective and reliably aligned with human intentions.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 96 of its 109 references that are in the map. First posted on arXiv 24 Aug 2025.
Builds on
96 of 109 references.
Closest ancestors · 4
Same direction, newest first
- 2024Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search14 Oct
- 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities5 Sep
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
- 2022Training language models to follow instructions with human feedback4 Mar
Other references · 92
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2017Deep reinforcement learning from human preferencescited by 3,912
38 more
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
- 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
- 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
- 2019Are Sixteen Heads Really Better than One?cited by 816
- 2023A Comprehensive Overview of Large Language Modelscited by 648
- 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
- 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 460
- 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
- 2020AdapterHub: A Framework for Adapting Transformerscited by 444
- 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
- 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
- 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 333
- 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
- 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
- 2023A Survey on Model Compression for Large Language Modelscited by 269
- 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
- 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 243
- 2023Inverse Scaling: When Bigger Isn't Bettercited by 207
- 2024Large Language Models for Education: A Survey and Outlookcited by 188
- 2024A Closer Look at the Limitations of Instruction Tuningcited by 77
- 2023Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMscited by 64
- 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunitiescited by 64
- 2023Maybe Only 0.5% Data is Needed: A Preliminary Exploration of Low Training Data Instruction Tuningcited by 58
- 2023A Bibliometric Review of Large Language Models Research from 2017 to 2023cited by 57
- 2023From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuningcited by 56
- 2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 48
- 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
- 2024Everybody Prune Now: Structured Pruning of LLMs with only Forward Passescited by 43
50 most cited of 92 All 92 (JSON)
Built on by
Citing papers per year: 2025: 4; 2026: 1.
5 papers.
- 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
- 2025Parameter-Efficient Fine-Tuning with Differential Privacy for Robust Instruction Adaptation in Large Language Modelscited by 5
- 2025GradientSpace: Unsupervised Data Clustering for Improved Instruction Tuningcited by 2
- 2025The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Modelscited by 2
- 2026Rethinking Agentic Reinforcement Learning In Large Language Modelscited by 1
- 2026Rethinking Agentic Reinforcement Learning In Large Language Models30 Apr 2026
- 2025The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Models26 Dec 2025
- 2025Parameter-Efficient Fine-Tuning with Differential Privacy for Robust Instruction Adaptation in Large Language Models7 Dec 2025
- 2025GradientSpace: Unsupervised Data Clustering for Improved Instruction Tuning7 Dec 2025
- 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey2 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025Aligning Instruction Tuning with Pre-training
- 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
- 2025MAIN: Mutual Alignment Is Necessary for instruction tuning
- 2025A Comprehensive Evaluation framework of Alignment Techniques for LLMs
- 2024CodecLM: Aligning Language Models with Tailored Synthetic Data
- 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
- 2024Improving the Robustness of Large Language Models via Consistency Alignment
- 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning