Reward-Guided Autoregressive Graph Generation for Efficient Multi-Agent Communication Topology Design
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
LLM-based Multi-Agent Systems (MAS) achieve strong performance on complex reasoning tasks by coordinating multiple agents, but at the cost of substantial token consumption. Recent work on automatic topology design, ARG-Designer, has reframed this problem as autoregressive graph generation. However, its training objective provides no explicit incentive for the model to generate sparse and efficient topologies. We address this limitation by introducing a Reward-Guided Autoregressive Graph Generation (RGA-Designer) inspired by Reinforcement Learning from Human Feedback (RLHF). We train a reward model that jointly captures task correctness and structural compactness, and then fine-tune the pretrained graph generator using the reward model as feedback. Our method preserves task accuracy at the level of ARG-Designer while reducing token consumption by an average of 20.5%.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 19 of its 30 references that are in the map. First posted on arXiv 20 Aug 2026.
Builds on
19 of 30 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2017Neural Message Passing for Quantum Chemistrycited by 3,487
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2014DeepWalk: Online Learning of Social Representationscited by 1,547
6 more
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
- 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
- 2016Solving General Arithmetic Word Problemscited by 360
- 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Graph-GRPO: Stabilizing Multi-Agent Topology Learning via Group Relative Policy Optimization
- 2025Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Models
- 2026RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Models
- 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks
- 2026Agent Q-Mix: Selecting the Right Action for LLM Multi-Agent Systems through Reinforcement Learning
- 2026AgentConductor: Topology Evolution for Multi-Agent Competition-Level Code Generation
- 2024Towards Generalizability of Multi-Agent Reinforcement Learning in Graphs with Recurrent Message Passing
- 2025Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systems