# MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models

arXiv: https://arxiv.org/abs/2402.01620 · HTML: https://mlsanity.com/paper/2402.01620/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 29 of 29)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://mlsanity.com/paper/2306.02707/) (2023)
- [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023)
- [FireAct: Toward Language Agent Fine-tuning](https://mlsanity.com/paper/2310.05915/) (2023)
- [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://mlsanity.com/paper/2311.01460/) (2023)
- [Orca 2: Teaching Small Language Models How to Reason](https://mlsanity.com/paper/2311.11045/) (2023)
- [Distilling Reasoning Capabilities into Smaller Language Models](https://mlsanity.com/paper/2212.00193/) (2022)
- [Directed Acyclic Graph Network for Conversational Emotion Recognition](https://mlsanity.com/paper/2105.12907/) (2021)
- [Agent Lumos: Unified and Modular Training for Open-Source Language Agents](https://mlsanity.com/paper/2311.05657/) (2023)
- [TinyGSM: achieving >80% on GSM8k with small language models](https://mlsanity.com/paper/2312.09241/) (2023)
- [Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models](https://mlsanity.com/paper/2304.13835/) (2023)
- [Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System](https://mlsanity.com/paper/1910.08381/) (2019)
- [Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification](https://mlsanity.com/paper/2012.13260/) (2020)
- [Dialogue Graph Modeling for Conversational Machine Reading](https://mlsanity.com/paper/2012.14827/) (2020)
- [Mixed Distillation Helps Smaller Language Model Better Reasoning](https://mlsanity.com/paper/2312.10730/) (2023)
- [Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization](https://mlsanity.com/paper/2306.09299/) (2023)

## Built on by (most cited first, 30 of 43)

- [ReSo: A Reward-driven Self-organizing LLM-based Multi-Agent System for Reasoning Tasks](https://mlsanity.com/paper/2503.02390/) (2025)
- [AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes](https://mlsanity.com/paper/2506.14728/) (2025)
- [Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence](https://mlsanity.com/paper/2410.11163/) (2024)
- [DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback](https://mlsanity.com/paper/2410.06215/) (2024)
- [Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems](https://mlsanity.com/paper/2502.04510/) (2025)
- [WebCoT: Enhancing Web Agent Reasoning by Reconstructing Chain-of-Thought in Reflection, Branching, and Rollback](https://mlsanity.com/paper/2505.20013/) (2025)
- [MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate](https://mlsanity.com/paper/2605.01347/) (2026)
- [Toward Efficient Agents: Memory, Tool learning, and Planning](https://mlsanity.com/paper/2601.14192/) (2026)
- [Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning](https://mlsanity.com/paper/2411.14497/) (2024)
- [Don't lie to your friends: Learning what you know from collaborative self-play](https://mlsanity.com/paper/2503.14481/) (2025)
- [AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent](https://mlsanity.com/paper/2602.03955/) (2026)
- [ChemToolAgent: The Impact of Tools on Language Agents for Chemistry Problem Solving](https://mlsanity.com/paper/2411.07228/) (2024)
- [Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Tasks](https://mlsanity.com/paper/2503.12029/) (2025)
- [Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancement](https://mlsanity.com/paper/2506.03541/) (2025)
- [Training with Harnesses: On-Policy Harness Self-Distillation for Complex Reasoning](https://mlsanity.com/paper/2605.08741/) (2026)
- [The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation](https://mlsanity.com/paper/2505.18759/) (2025)
- [QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models](https://mlsanity.com/paper/2405.13014/) (2024)
- [Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AI](https://mlsanity.com/paper/2510.16720/) (2025)
- [AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generation](https://mlsanity.com/paper/2510.19361/) (2025)
- [From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?](https://mlsanity.com/paper/2604.01608/) (2026)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)
- [ORPO-Distill: Mixed-Policy Preference Optimization for Cross-Architecture LLM Distillation](https://mlsanity.com/paper/2509.25100/) (2025)
- [Learning Steerable Clarification Policies with Collaborative Self-play](https://mlsanity.com/paper/2512.04068/) (2025)
- [Spatial Audio Processing with Large Language Model on Wearable Devices](https://mlsanity.com/paper/2504.08907/) (2025)
- [DeepKD: A Deeply Decoupled and Denoised Knowledge Distillation Trainer](https://mlsanity.com/paper/2505.15133/) (2025)
- [SMAGDi: Socratic Multi Agent Interaction Graph Distillation for Efficient High Accuracy Reasoning](https://mlsanity.com/paper/2511.05528/) (2025)
- [Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges](https://mlsanity.com/paper/2607.26212/) (2026)
- [The Single-Multi Evolution Loop for Self-Improving Model Collaboration Systems](https://mlsanity.com/paper/2602.05182/) (2026)
- [Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2609.23697/) (2026)
- [Agentic AI: The Era of Semantic Decoding](https://mlsanity.com/paper/2403.14562/) (2024)

## Abstract (arXiv)

Multi-agent interactions between Large Language Model (LLM) agents have shown major improvements on diverse reasoning tasks. However, these involve long generations from multiple models across several rounds, making them expensive. Moreover, these multi-agent approaches fail to provide a final, single model for efficient inference. To address this, we introduce MAGDi, a new method for structured distillation of the reasoning interactions between multiple LLMs into smaller LMs. MAGDi teaches smaller models by representing multi-agent interactions as graphs, augmenting a base student model with a graph encoder, and distilling knowledge using three objective functions: next-token prediction, a contrastive loss between correct and incorrect reasoning, and a graph-based objective to model the interaction structure. Experiments on seven widely used commonsense and math reasoning benchmarks show that MAGDi improves the reasoning capabilities of smaller models, outperforming several methods that distill from a single teacher and multiple teachers. Moreover, MAGDi also demonstrates an order of magnitude higher efficiency over its teachers. We conduct extensive analyses to show that MAGDi (1) enhances the generalizability to out-of-domain tasks, (2) scales positively with the size and strength of the base student model, and (3) obtains larger improvements (via our multi-teacher training) when applying self-consistency -- an inference technique that relies on model diversity.