mlsanityData to 7 Oct 2026

AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes

arXiv 2506.14728 ↗First posted 17 Jun 2025cs.AIcited by 28

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

While knowledge distillation has become a mature field for compressing large language models (LLMs) into smaller ones by aligning their outputs or internal representations, the distillation of LLM-based agents, which involve planning, memory, and tool use, remains relatively underexplored. Existing agent distillation methods typically replay full teacher trajectories or imitate step-by-step teacher tool usage, but they often struggle to train student agents to dynamically plan and act in novel environments. We propose AgentDistill, a novel, training-free agent distillation framework that enables efficient and scalable knowledge transfer via direct reuse of Model-Context-Protocols (MCPs), which are structured and reusable task-solving modules autonomously generated by teacher agents. The reuse of these distilled MCPs enables student agents to generalize their capabilities across domains and solve new problems with minimal supervision or human intervention. Experiments on biomedical and mathematical benchmarks demonstrate that our distilled student agents, built on small language models, can achieve performance comparable to advanced systems using large LLMs such as OctoTools (GPT-4o), highlighting the effectiveness of our framework in building scalable and cost-efficient intelligent agents.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 46 of its 51 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

46 of 51 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation29 May
  2. 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution26 May
  3. 2025Distilling LLM Agent into Small Models with Retrieval and Code Tools23 May
  4. 2025Structured Agent Distillation for Large Language Model20 May
  5. 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning16 Feb
  6. 2024Sub-goal Distillation: A Method to Improve Small Language Agents4 May
  7. 2024MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models2 Feb
  8. 2024Knowledge Distillation from Language-Oriented to Emergent Communication for Multi-Agent Remote Control23 Jan
  9. 2023AutoAgents: A Framework for Automatic Agent Generation29 Sep

Other references · 37

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  4. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  5. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 502
  6. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  7. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  8. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  9. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  10. 2022Large Language Models Are Reasoning Teacherscited by 318
  11. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  12. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
25 more
  1. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 155
  2. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Stepcited by 144
  3. 2022Learning by Distilling Contextcited by 128
  4. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 123
  5. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 103
  6. 2024ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learningcited by 63
  7. 2025ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templatescited by 62
  8. 2025ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoningcited by 41
  9. 2024FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Modelscited by 39
  10. 2022In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Modelscited by 36
  11. 2024AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligencecited by 36
  12. 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 33
  13. 2024SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correctioncited by 32
  14. 2024ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoningcited by 31
  15. 2022DISCO: Distilling Counterfactuals with Large Language Modelscited by 29
  16. 2024Long Term Memory: The Foundation of AI Self-Evolutioncited by 29
  17. 2024Learning to Maximize Mutual Information for Chain-of-Thought Distillationcited by 23
  18. 2024Keypoint-based Progressive Chain-of-Thought Distillation for LLMscited by 22
  19. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  20. 2023GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Modelcited by 15
  21. 2025Skip-Thinking: Chunk-wise Chain-of-Thought Distillation Enable Smaller Language Models to Reason Better and Fastercited by 15
  22. 2024Embodied CoT Distillation From LLM To Off-the-shelf Agentscited by 9
  23. 2025On Path to Multimodal Historical Reasoning: HistBench and HistAgentcited by 5
  24. 2024Efficient LLM Context Distillationcited by 4
  25. 2024In-Context Learning Distillation for Efficient Few-Shot Fine-Tuningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 18.

28 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  3. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 99
  4. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 81
  5. 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agentscited by 37
  6. 2026On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agentscited by 21
  7. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 17
  8. 2026SOD: Step-wise On-policy Distillation for Small Language Model Agentscited by 17
  9. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  10. 2025AgentMaster: A Multi-Agent Conversational Framework Using A2A and MCP Protocols for Multimodal Information Retrieval and Analysiscited by 13
  11. 2025LatentEvolve: Self-Evolving Test-Time Scaling in Latent Spacecited by 8
  12. 2026PhGPO: Pheromone-Guided Policy Optimization for Long-Horizon Tool Planningcited by 6
16 more
  1. 2026From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?cited by 6
  2. 2026Learning Agent Routing From Early Experiencecited by 3
  3. 2026Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memorycited by 3
  4. 2025LLM×MapReduce-V3: Enabling Interactive In-Depth Survey Generation through a MCP-Driven Hierarchically Modular Agent Systemcited by 2
  5. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  6. 2026Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM Collectivescited by 2
  7. 2026SKILL-DISCO: Distilling and Compiling Agent Traces into Reusable Procedural Skillscited by 2
  8. 2026SKILL-KD: Contrastive Skill Distillation for LLM Agentscited by 2
  9. 2026AQuA: Recursively Self-Improving Quantitative Trading Research Agentscited by 2
  10. 2025MENTOR: Reinforcement Learning via Flexible Teacher-Optimized Rewards for Tool-Use Distillationcited by 1
  11. 2026TED: Training-Free Experience Distillation for Multimodal Reasoningcited by 1
  12. 2026From Trajectories to Prefixes: Reusing Teacher Trajectories via Replayed Prefixes and Online Continuationcited by 1
  13. 2026AgentBrew: Lifelong Knowledge Brewing from Strong Teachers to Weak LLM Agentscited by 0
  14. 2026Small Reasoning Models are Instruction Followers in Function Callingcited by 0
  15. 2026Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnessescited by 0
  16. 2026Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs
  2. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning
  3. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs
  4. 2026Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent
  5. 2026Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers
  6. 2026Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation
  7. 2026Harness-Aware Distillation for Small Language Model Agents
  8. 2024Offline Training of Language Model Agents with Functions as Learnable Weights