mlsanityData to 7 Oct 2026

From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?

arXiv 2604.01608 ↗First posted 2 Apr 2026cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Multi-agent systems (MAS) for structured data-science tasks externalize analytical control through workflows spanning stages, tools, shared state, verification, and repair. Distilling such workflows into a single-agent skill can reduce orchestration overhead, but it remains unclear which workflow components should cross the control boundary. We distinguish capability resources, which expand what an agent can do, from pipeline guidance, which constrains which solutions it explores. On the same causal-estimation instances, adding task-qualified source pipeline guidance to a capability-matched skill changes normalized utility by +19.6 points under method-selection accuracy but -10.3 points under numerical error. To explain this reversal, we introduce Behavior-Outcome Freedom (F), a pre-synthesis diagnostic of signed behavior-outcome rank mismatch, and formalize its candidate-conditional role through Signed Anchor-Rank Transfer. Motivated by this mechanism, we propose AdaSkill, which preserves validated capability resources, removes runtime orchestration, and conditionally inherits pipeline guidance using a calibrated rule over F. Across 16 capability-matched interventions, the native-scale Full-minus-Discard effect decreases across the continuous F scale (r = -0.80, p < 0.001), while a 15-treatment atomic sweep localizes the reversal to pipeline guidance. Across 11 datasets spanning four structured data-science task families, AdaSkill combines strong task performance with substantially lower deployment overhead.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 32 of its 61 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

32 of 61 references.

References · 32

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  8. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  10. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  11. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 745
  12. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
20 more
  1. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  2. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  3. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 223
  4. 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skillscited by 161
  5. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 119
  6. 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolutioncited by 105
  7. 2026EvoSkill: Automated Skill Discovery for Multi-Agent Systemscited by 96
  8. 2024MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Modelscited by 41
  9. 2026When Single-Agent with Skills Replace Multi-Agent Systems and When They Failcited by 34
  10. 2025AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxescited by 28
  11. 2026Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baselinecited by 18
  12. 2025One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIPcited by 15
  13. 2026AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agentcited by 12
  14. 2024Exploring Multi-Modal Data with Tool-Augmented LLM Agents for Precise Causal Discoverycited by 11
  15. 2025CLIP-Guided Backdoor Defense through Entropy-Based Poisoned Dataset Separationcited by 9
  16. 2025Breaking the Stealth-Potency Trade-off in Clean-Image Backdoors with Generative Trigger Optimizationcited by 7
  17. 2026APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQLcited by 7
  18. 2025SMAGDi: Socratic Multi Agent Interaction Graph Distillation for Efficient High Accuracy Reasoningcited by 3
  19. 2025Double Distillation Network for Multi-Agent Reinforcement Learningcited by 2
  20. 2025FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Datacited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolutioncited by 18
  2. 2026Contextual Agentic Memory is a Memo, Not True Memorycited by 5
  3. 2026LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioceptioncited by 3
  4. 2026Reviewer Scores Are Not Comparable Across Research Areas in ML Peer Reviewcited by 2
  5. 2026Attributing Structured-Output Gains in Function Calling: Interface Alignment versus Procedural Transfercited by 0
  6. 2026Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnessescited by 0