mlsanityData to 14 Oct 2026

SeDi-Instruct: Enhancing Alignment of Language Models through Self-Directed Instruction Generation

arXiv 2502.04774 ↗First posted 7 Feb 2025cs.CLcited by 3

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

The rapid evolution of Large Language Models (LLMs) has enabled the industry to develop various AI-based services. Instruction tuning is considered essential in adapting foundation models for target domains to provide high-quality services to customers. A key challenge in instruction tuning is obtaining high-quality instruction data. Self-Instruct, which automatically generates instruction data using ChatGPT APIs, alleviates the data scarcity problem. To improve the quality of instruction data, Self-Instruct discards many of the instructions generated from ChatGPT, even though it is inefficient in terms of cost owing to many useless API calls. To generate high-quality instruction data at a low cost, we propose a novel data generation framework, Self-Direct Instruction generation (SeDi-Instruct), which employs diversity-based filtering and iterative feedback task generation. Diversity-based filtering maintains model accuracy without excessively discarding low-quality generated instructions by enhancing the diversity of instructions in a batch. This reduces the cost of synthesizing instruction data. The iterative feedback task generation integrates instruction generation and training tasks and utilizes information obtained during the training to create high-quality instruction sets. Our results show that SeDi-Instruct enhances the accuracy of AI models by 5.2%, compared with traditional methods, while reducing data generation costs by 36%.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 13 of its 39 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

13 of 39 references.

References · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  5. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  6. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  7. 2023Instruction Tuning with GPT-4cited by 686
  8. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  9. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  10. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 264
  11. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 151
  12. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 45
1 more
  1. 2019Batch Active Learning Using Determinantal Point Processescited by 43

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 51
  2. 2026TTVS: Boosting Self-Exploring Reinforcement Learning via Test-time Variational Synthesiscited by 1
  3. 2026Evolutionary Task Discovery: Advancing Reasoning Frontiers via Skill Composition and Complexity Scalingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions
  2. 2023Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
  3. 2024SelfCodeAlign: Self-Alignment for Code Generation
  4. 2023Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs
  5. 2024Self-Judge: Selective Instruction Following with Alignment Self-Evaluation
  6. 2025InstructLR: A Scalable Approach to Create Instruction Dataset for Under-Resourced Languages
  7. 2024Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search
  8. 2024A New Pipeline For Generating Instruction Dataset via RAG and Self Fine-Tuning