mlsanityData to 14 Oct 2026

Self-Instruct: Aligning Language Models with Self-Generated Instructions

arXiv 2212.10560 ↗First posted 20 Dec 2022cs.CL cs.AIcited by 2,724

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

also in 1 more
Abstract (arXiv)

Large "instruction-tuned" language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless, they depend heavily on human-written instruction data that is often limited in quantity, diversity, and creativity, therefore hindering the generality of the tuned model. We introduce Self-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off their own generations. Our pipeline generates instructions, input, and output samples from a language model, then filters invalid or similar ones before using them to finetune the original model. Applying our method to the vanilla GPT3, we demonstrate a 33% absolute improvement over the original model on Super-NaturalInstructions, on par with the performance of InstructGPT-001, which was trained with private user data and human annotations. For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin, leaving only a 5% absolute gap behind InstructGPT-001. Self-Instruct provides an almost annotation-free method for aligning pre-trained language models with instructions, and we release our large synthetic dataset to facilitate future studies on instruction tuning. Our code and data are available at https://github.com/yizhongw/self-instruct.

Key facts. Cited by 2,724 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 48 of its 66 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

48 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 47

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  8. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  9. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  10. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  12. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
35 more
  1. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  2. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  3. 2022Large Language Models Can Self-Improvecited by 664
  4. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  5. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  6. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  7. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  8. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  9. 2022Teaching Small Language Models to Reasoncited by 284
  10. 2022Generating Sequences by Learning to Self-Correctcited by 277
  11. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  12. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 202
  13. 2018Constituency Parsing with a Self-Attentive Encodercited by 199
  14. 2021Generating Datasets with Pretrained Language Modelscited by 193
  15. 2022Fine-tuned Language Models are Continual Learnerscited by 156
  16. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  17. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  18. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  19. 2021FILM: Following Instructions in Language with Modular Methodscited by 121
  20. 2018Multilingual Constituency Parsing with Self-Attention and Pre-Trainingcited by 117
  21. 2020Self-training Improves Pre-training for Natural Language Understandingcited by 96
  22. 2022Self-Training: A Surveycited by 86
  23. 2020Learning from Task Descriptionscited by 78
  24. 2021Towards Zero-Label Language Learningcited by 76
  25. 2022Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writingcited by 71
  26. 2022Prompt Consistency for Zero-Shot Task Generalizationcited by 67
  27. 2022Unsupervised Cross-Task Generalization via Retrieval Augmentationcited by 42
  28. 2022InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuningcited by 39
  29. 2022Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learningcited by 36
  30. 2022ConTinTin: Continual Learning from Task Instructionscited by 26
  31. 2022Pre-trained Language Models Can be Fully Zero-Shot Learnerscited by 22
  32. 2022How Many Data Samples is an Additional Instruction Worth?cited by 16
  33. 2022BioTABQA: Instruction Learning for Biomedical Table Question Answeringcited by 13
  34. 2022Leveraging QA Datasets to Improve Generative Data Augmentationcited by 12
  35. 2022One-Shot Learning from a Demonstration with Hierarchical Latent Languagecited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 616; 2024: 898; 2025: 654; 2026: 554.

2,724 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2023A Survey of Large Language Modelscited by 2,576
  10. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  11. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  12. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
38 more
  1. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  2. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  5. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  7. 2023LIMA: Less Is More for Alignmentcited by 1,100
  8. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  9. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  10. 2023A Survey on Multimodal Large Language Modelscited by 909
  11. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  12. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  13. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  14. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  15. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  16. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  17. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  18. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  19. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  20. 2022A Survey on In-context Learningcited by 721
  21. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  22. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  23. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  24. 2023Instruction Tuning with GPT-4cited by 686
  25. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  26. 2024Self-Rewarding Language Modelscited by 681
  27. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  28. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  29. 2024A Survey on Large Language Models for Code Generationcited by 658
  30. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  31. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  32. 2023A Comprehensive Overview of Large Language Modelscited by 648
  33. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  34. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  35. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  36. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  37. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
  38. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575

50 most cited of 2,724 All 2,724 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025SeDi-Instruct: Enhancing Alignment of Language Models through Self-Directed Instruction Generation
  2. 2025Training with Pseudo-Code for Instruction Following
  3. 2026GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
  4. 2023Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
  5. 2024Teaching Language Models to Self-Improve by Learning from Language Feedback
  6. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models
  7. 2024From Instance Training to Instruction Learning: Task Adapters Generation from Instructions
  8. 2023InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuning