mlsanityData to 7 Oct 2026

MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching

arXiv 2506.02689 ↗First posted 3 Jun 2025cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Instruction fine-tuning is crucial in NLP tasks, enhancing pretrained models' instruction-following capabilities and task-specific performance. However, obtaining high-quality fine-tuning data for large models is challenging due to data collection difficulties and high production costs. To address this, we propose MASTER, a novel data augmentation method that enriches original data through interactions among multiple agents with varying cognitive levels. We simulate three pedagogically grounded teaching scenarios, leveraging multi-agent conversations to generate high-quality teacher-student interaction data. Utilizing MASTER, we construct BOOST-QA, a fine-tuning dataset augmented from existing datasets like Orca-Math-200k, ProcQA, and OpenHermes2.5. Experiments show that models fine-tuned with BOOST-QA perform excellently across multiple benchmarks, demonstrating strong multitask generalization. Notably, MASTER significantly improves models' reasoning abilities in complex tasks, providing valuable insights for future research.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 40 of its 57 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

40 of 57 references.

References · 40

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2024Qwen2.5 Technical Reportcited by 4,713
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  11. 2023Mistral 7Bcited by 3,065
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
28 more
  1. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  2. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  3. 2019Relational Knowledge Distillationcited by 873
  4. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  5. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  6. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  7. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  8. 2017Deep Mutual Learningcited by 621
  9. 2019Similarity-Preserving Knowledge Distillationcited by 507
  10. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  11. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  12. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  13. 2024Simulating Classroom Education with LLM-Empowered Agentscited by 125
  14. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
  15. 2024Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Samplingcited by 85
  16. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  17. 2024Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulationcited by 69
  18. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agentscited by 65
  19. 2020CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understandingcited by 41
  20. 2021FlipDA: Effective and Robust Data Augmentation for Few-Shot Learningcited by 32
  21. 2024TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Datacited by 28
  22. 2025Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinementcited by 17
  23. 2025Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoningcited by 14
  24. 2025Generator-Assistant Stepwise Rollback Framework for Large Language Model Agentcited by 13
  25. 2024ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Searchcited by 12
  26. 2025TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillationcited by 12
  27. 2025The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agentscited by 7
  28. 2024PGA-SciRE: Harnessing LLM on Data Augmentation for Enhancing Scientific Relation Extractioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026Mem2Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillationcited by 10
  2. 2025Toward Automated Robustness Evaluation of Mathematical Reasoningcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2025EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework
  2. 2025Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment
  3. 2024Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning
  4. 2024CoEvol: Constructing Better Responses for Instruction Finetuning through Multi-Agent Cooperation
  5. 2026EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners
  6. 2024Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents
  7. 2025CoDAE: Adapting Large Language Models for Education via Chain-of-Thought Data Augmentation
  8. 2024AgentInstruct: Toward Generative Teaching with Agentic Flows