mlsanityData to 7 Oct 2026

I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm

arXiv 2408.08072 ↗First posted 15 Aug 2024cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have achieved significant advancements, however, the common learning paradigm treats LLMs as passive information repositories, neglecting their potential for active learning and alignment. Some approaches train LLMs using their own generated synthetic data, exploring the possibility of active alignment. However, there is still a huge gap between these one-time alignment methods and the continuous automatic alignment of humans. In this paper, we introduce I-SHEEP, an Iterative Self-EnHancEmEnt Paradigm.This human-like paradigm enables LLMs to continuously self-align from scratch with nothing. Compared to the one-time alignment method Dromedary citesun2023principledriven, which refers to the first iteration in this paper, I-SHEEP can significantly enhance capacities on both Qwen and Llama models. I-SHEEP achieves a maximum relative improvement of 78.2% in the Alpaca Eval, 24.0% in the MT Bench, and an absolute increase of 8.88% in the IFEval accuracy over subsequent iterations in Qwen-1.5 72B model. Additionally, I-SHEEP surpasses the base model in various standard benchmark generation tasks, achieving an average improvement of 24.77% in code generation tasks, 12.04% in TrivialQA, and 20.29% in SQuAD. We also provide new insights based on the experiment results. Our codes, datasets, and models are available at https://anonymous.4open.science/r/I-SHEEP.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 42 of its 58 references that are in the map. First posted on arXiv 15 Aug 2024.

Builds on

42 of 58 references.

References · 42

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
30 more
  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  4. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  5. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  6. 2023LIMA: Less Is More for Alignmentcited by 1,080
  7. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  8. 2022Large Language Models Can Self-Improvecited by 656
  9. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  10. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 354
  11. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
  12. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  13. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  14. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  15. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  16. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
  17. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  18. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 134
  19. 2023MoDS: Model-oriented Data Selection for Instruction Tuningcited by 107
  20. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 69
  21. 2023Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculumcited by 64
  22. 2024Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimationcited by 53
  23. 2024COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuningcited by 45
  24. 2023WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuningcited by 42
  25. 2024Genie: Achieving Human Parity in Content-Grounded Datasets Generationcited by 31
  26. 2023SELF: Self-Evolution with Language Feedbackcited by 24
  27. 2024Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Levelcited by 22
  28. 2024Step-On-Feet Tuning: Scaling Self-Alignment of LLMs via Bootstrappingcited by 14
  29. 2024How Do Humans Write Code? Large Models Do It the Same Way Toocited by 11
  30. 2024Kun: Answer Polishment for Chinese Self-Alignment with Instruction Back-Translationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 17; 2026: 5.

27 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  2. 2025Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challengescited by 30
  3. 2025Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinementcited by 17
  4. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Modelscited by 16
  5. 2025Curiosity-Driven Reinforcement Learning from Human Feedbackcited by 14
  6. 2024Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Samplingcited by 13
  7. 2025MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Trainingcited by 12
  8. 2025ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflectioncited by 11
  9. 2025RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Datacited by 11
  10. 2025AIR: Complex Instruction Generation via Automatic Iterative Refinementcited by 10
  11. 2025LLMs for Customized Marketing Content Generation and Evaluation at Scalecited by 9
  12. 2025A Survey on Efficient Large Language Model Training: From Data-centric Perspectivescited by 9
15 more
  1. 2025SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoningcited by 8
  2. 2025Aligning Instruction Tuning with Pre-trainingcited by 7
  3. 2025The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillationcited by 7
  4. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7
  5. 2024Language Models as Continuous Self-Evolving Data Engineerscited by 6
  6. 2025Enterprise Large Language Model Evaluation Benchmarkcited by 5
  7. 2025ACT as Human: Multimodal Large Language Model Data Annotation with Critical Thinkingcited by 3
  8. 2024MentalArena: Self-play Training of Language Models for Diagnosis and Treatment of Mental Health Disorderscited by 1
  9. 2025AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Researchcited by 1
  10. 2026SEIF: Self-Evolving Reinforcement Learning for Instruction Followingcited by 1
  11. 2025SeaPO: Strategic Error Amplification for Robust Preference Optimization of Large Language Modelscited by 0
  12. 2026Large Language Models Can Take False First Steps at Inference-time Planningcited by 0
  13. 2026Maximizing Mutual Information Between Prompt and Response Improves LLM Performance With No Additional Datacited by 0
  14. 2026Trust Functions: Near-Lossless Weak-to-Strong Generalization by Learning When to Trust the Weak Teachercited by 0
  15. 2026Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Human-Instruction-Free LLM Self-Alignment with Limited Samples
  2. 2025A Survey on Training-free Alignment of Large Language Models
  3. 2024Towards Scalable Automated Alignment of LLMs: A Survey
  4. 2025A Comprehensive Evaluation framework of Alignment Techniques for LLMs
  5. 2024Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillation
  6. 2025RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars
  7. 2024Aligners: Decoupling LLMs and Alignment
  8. 2024Efficient Alignment of Large Language Models via Data Sampling