mlsanityData to 7 Oct 2026

Self-Alignment with Instruction Backtranslation

arXiv 2308.06259 ↗First posted 11 Aug 2023cs.CLcited by 149

Not on the map. Why?

Cited by 31 in Synthetic data and self-improvement for LLMs, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a scalable method to build a high quality instruction following language model by automatically labelling human-written text with corresponding instructions. Our approach, named instruction backtranslation, starts with a language model finetuned on a small amount of seed data, and a given web corpus. The seed model is used to construct training examples by generating instruction prompts for web documents (self-augmentation), and then selecting high quality examples from among these candidates (self-curation). This data is then used to finetune a stronger model. Finetuning LLaMa on two iterations of our approach yields a model that outperforms all other LLaMa-based models on the Alpaca leaderboard not relying on distillation data, demonstrating highly effective self-alignment.

Key facts. Cited by 149 papers in the map (data through 7 Oct 2026). Builds on 34 of its 51 references that are in the map. First posted on arXiv 11 Aug 2023.

Builds on

34 of 51 references.

References · 34

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
22 more
  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  7. 2023LIMA: Less Is More for Alignmentcited by 1,080
  8. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  9. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  10. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  11. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  12. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  13. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  14. 2023Instruction Tuning with GPT-4cited by 676
  15. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  16. 2022Self-critiquing models for assisting human evaluatorscited by 372
  17. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  18. 2023The False Promise of Imitating Proprietary LLMscited by 254
  19. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  20. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  21. 2019Tagged Back-Translationcited by 76
  22. 2023Self-QA: Unsupervised Knowledge Guided Language Model Alignmentcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 19; 2024: 72; 2025: 45; 2026: 13.

149 papers.

Sort:

By area:

  1. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  2. 2024Self-Rewarding Language Modelscited by 654
  3. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 354
  4. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  5. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  6. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
  7. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  8. 2024Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modelingcited by 156
  9. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  10. 2024MAmmoTH2: Scaling Instructions from the Webcited by 143
  11. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 142
  12. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 140
38 more
  1. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 123
  2. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  3. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  4. 2023HuatuoGPT-II, One-stage Training for Medical Adaption of LLMscited by 91
  5. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  6. 2024Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Modelscited by 83
  7. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 81
  8. 2023Data Diversity Matters for Robust Instruction Tuningcited by 76
  9. 2024A Survey on Data Selection for LLM Instruction Tuningcited by 76
  10. 2025Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judgecited by 75
  11. 2023Alignment for Honestycited by 72
  12. 2023Table-GPT: Table-tuned GPT for Diverse Table Taskscited by 69
  13. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 69
  14. 2024Self-Generated Critiques Boost Reward Modeling for Language Modelscited by 69
  15. 2024Self-Exploring Language Models: Active Preference Elicitation for Online Alignmentcited by 63
  16. 2024SelfCodeAlign: Self-Alignment for Code Generationcited by 61
  17. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 56
  18. 2023SALMON: Self-Alignment with Instructable Reward Modelscited by 54
  19. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  20. 2024Bootstrapping Language Models with DPO Implicit Rewardscited by 53
  21. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
  22. 2024Reverse Training to Nurse the Reversal Cursecited by 48
  23. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
  24. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 47
  25. 2024FLAME: Factuality-Aware Alignment for Large Language Modelscited by 46
  26. 2024Theoretical Analysis of Weak-to-Strong Generalizationcited by 46
  27. 2025Scaling Laws of Synthetic Data for Language Modelscited by 44
  28. 2024MUSTARD: Mastering Uniform Synthesis of Theorem and Proof Datacited by 43
  29. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 43
  30. 2024Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scalecited by 43
  31. 2024Instruction Tuning With Loss Over Instructionscited by 40
  32. 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 39
  33. 2024Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B Peoplecited by 38
  34. 2024SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literaturecited by 37
  35. 2025Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translationcited by 37
  36. 2024SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflectioncited by 34
  37. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 31
  38. 2023MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Followingcited by 30

50 most cited of 149 All 149 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Better Alignment with Instruction Back-and-Forth Translation
  2. 2024Align2LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation
  3. 2024Self-Judge: Selective Instruction Following with Alignment Self-Evaluation
  4. 2024REInstruct: Building Instruction Data from Unlabeled Corpus
  5. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
  6. 2024Toward General Instruction-Following Alignment for Retrieval-Augmented Generation
  7. 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
  8. 2025SeDi-Instruct: Enhancing Alignment of Language Models through Self-Directed Instruction Generation