mlsanityData to 14 Oct 2026

Tag-Evol: Achieving Efficient Instruction Evolving via Tag Injection

arXiv 2505.24165 ↗First posted 30 May 2025cs.CLcited by 0

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

Evol-Instruct has made significant improvements as a data synthesis method in several areas. Existing methods typically rely on a fixed set of strategies to evolve, which require manual design and are monolithic in form. In addition, iterative evolution also makes the acquisition of hard samples expensive. In view of this, we propose the Tag-Evol framework, a more diverse and efficient instruction evolving method. Specifically, Tag-Evol uses diverse and specific knowledge tags as strategies to achieve controlled evolution by injecting different combinations of tags into the original instructions. Experiments with multiple backbones in diverse domain benchmarks show that the proposed method generates significantly better evolved data than other methods. Furthermore, we conduct a thorough analysis of the evolved data, demonstrating that Tag-Evol is not only efficient but also generates more diverse and challenging data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 17 of its 27 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

17 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec

Other references · 16

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  9. 2023LIMA: Less Is More for Alignmentcited by 1,100
  10. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  11. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  12. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
4 more
  1. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  2. 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 205
  3. 2023TinyGSM: achieving >80% on GSM8k with small language modelscited by 70
  4. 2024Smaller Language Models Are Better Instruction Evolverscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search
  2. 2023Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
  3. 2025InsBank: Evolving Instruction Subset for Ongoing Alignment
  4. 2025Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction
  5. 2024AMR-Evol: Adaptive Modular Response Evolution Elicits Better Knowledge Distillation for Large Language Models in Code Generation
  6. 2026EvolSQL: Structure-Aware Evolution for Scalable Text-to-SQL Data Synthesis
  7. 2025MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning
  8. 2024Toward General Instruction-Following Alignment for Retrieval-Augmented Generation