# OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models

arXiv: https://arxiv.org/abs/2601.12996 · HTML: https://mlsanity.com/paper/2601.12996/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 19 Jan 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)
- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 26 of 26)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [Solving General Arithmetic Word Problems](https://mlsanity.com/paper/1608.01413/) (2016)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [Efficient Graph Generation with Graph Recurrent Attention Networks](https://mlsanity.com/paper/1910.00760/) (2019)
- [Multi-agent Architecture Search via Agentic Supernet](https://mlsanity.com/paper/2502.04180/) (2025)
- [A Survey of Multi-Agent Deep Reinforcement Learning with Communication](https://mlsanity.com/paper/2203.08975/) (2022)
- [Self-Evolving Multi-Agent Collaboration Networks for Software Development](https://mlsanity.com/paper/2410.16946/) (2024)
- [Assemble Your Crew: Automatic Multi-agent Communication Topology Design via Autoregressive Graph Generation](https://mlsanity.com/paper/2507.18224/) (2025)
- [ARC: A Generalist Graph Anomaly Detector with In-Context Learning](https://mlsanity.com/paper/2405.16771/) (2024)
- [Pard: Permutation-Invariant Autoregressive Diffusion for Graph Generation](https://mlsanity.com/paper/2402.03687/) (2024)
- [A Label-Free Heterophily-Guided Approach for Unsupervised Graph Fraud Detection](https://mlsanity.com/paper/2502.13308/) (2025)
- [Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach](https://mlsanity.com/paper/2506.13083/) (2025)
- [A Survey of Generalization of Graph Anomaly Detection: From Transfer Learning to Foundation Models](https://mlsanity.com/paper/2509.06609/) (2025)
- [Correcting False Alarms from Unseen: Adapting Graph Anomaly Detectors at Test Time](https://mlsanity.com/paper/2511.07023/) (2025)
- [Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality Estimation](https://mlsanity.com/paper/2407.19944/) (2024)
- [FreeGAD: A Training-Free yet Effective Approach for Graph Anomaly Detection](https://mlsanity.com/paper/2508.10594/) (2025)
- [Overcoming Order in Autoregressive Graph Generation](https://mlsanity.com/paper/2402.03387/) (2024)

## Built on by (most cited first, 14 of 14)

- [DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning](https://mlsanity.com/paper/2603.16459/) (2026)
- [Towards One-for-All Anomaly Detection for Tabular Data](https://mlsanity.com/paper/2603.14407/) (2026)
- [FedCIGAR: A Personalized Reconstruction Approach for Federated Graph-level Anomaly Detection](https://mlsanity.com/paper/2605.09428/) (2026)
- [Towards Anomaly Detection on Relational Data](https://mlsanity.com/paper/2606.18621/) (2026)
- [Beyond a Single Perspective: Text Anomaly Detection with Multi-View Language Representations](https://mlsanity.com/paper/2601.17786/) (2026)
- [LEMON: Learning Executable Multi-Agent Orchestration via Counterfactual Reinforcement Learning](https://mlsanity.com/paper/2605.14483/) (2026)
- [Discovering Efficient and Explainable Communication Topologies for LLM-based Multi-Agent Systems via Causal Inference](https://mlsanity.com/paper/2608.12921/) (2026)
- [PaSta: Noisy Node Classification with Partial Label Learning](https://mlsanity.com/paper/2608.25365/) (2026)
- [SIM: Subspace Interaction-based Method for Token-Level Text Anomaly Detection](https://mlsanity.com/paper/2609.08200/) (2026)
- [CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents](https://mlsanity.com/paper/2609.18779/) (2026)
- [DiFA: Dual Evidence Fusion and Aggregation for Token-Level Text Anomaly Detection](https://mlsanity.com/paper/2609.22136/) (2026)
- [Beyond Solo and Consistency: Vindicating Multi-Agent Debate via Conditional Progressive Pruning](https://mlsanity.com/paper/2609.33974/) (2026)
- [Token-Efficient Multi-Agent Collaboration via System One-Guided Computational Division of Labor](https://mlsanity.com/paper/2610.08155/) (2026)
- [MAS-Gate: Unlocking Multi-Agent System Lifecycle via Node-Centric Quantification](https://mlsanity.com/paper/2610.17613/) (2026)

## Abstract (arXiv)

Multi-Agent Systems (MAS) offer a powerful paradigm for solving complex problems, yet their performance is critically dependent on the design of their underlying collaboration topology. As MAS become increasingly deployed in web services (e.g., search engines), designing adaptive topologies for diverse cross-domain user queries becomes essential. Current graph learning-based design methodologies often adhere to a "one-for-one" paradigm, where a specialized model is trained for each specific task domain. This approach suffers from poor generalization to unseen domains and fails to leverage shared structural knowledge across different tasks. To address this, we propose OFA-TAD, a one-for-all framework that generates adaptive collaboration graphs for any task described in natural language through a single universal model. Our approach integrates a Task-Aware Graph State Encoder (TAGSE) that filters task-relevant node information via sparse gating, and a Mixture-of-Experts (MoE) architecture that dynamically selects specialized sub-networks to drive node and edge prediction. We employ a three-stage training strategy: unconditional pre-training on canonical topologies for structural priors, large-scale conditional pre-training on LLM-generated datasets for task-topology mappings, and supervised fine-tuning on empirically validated graphs. Experiments across six diverse benchmarks show that OFA-TAD significantly outperforms specialized one-for-one models, generating highly adaptive MAS topologies. Code: https://github.com/Shiy-Li/OFA-MAS.