MMoA: An AI-Agent framework with recurrence for Memoried Mixure-of-Agent
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
The Mixture-of-Agents (MoA) framework has shown promise in improving large language model (LLM) performance by aggregating outputs from multiple agents. However, existing MoA systems often rely on static routers that do not fully capture temporal and contextual dependencies across aggregation layers. To address this limitation, we propose MMoA, a recurrent MoA architecture that integrates LSTM-based gating into the agent selection process. The recurrence router adaptively modulates agent contributions based on both current inputs and historical routing decisions, enabling more context-aware aggregation. We evaluate MMoA on standard instruction-following benchmarks, including AlpacaEval 2.0, MT-Bench, and Arena-Hard. The results show that MMoA achieves comparable accuracy to traditional MoA while reducing computational overhead by dynamically activating fewer agents. For example, on AlpacaEval 2.0, MMoA achieves a win rate of 58.0%, compared with 59.8% for MoA, while improving runtime efficiency by up to 4.6%. These results suggest that MMoA provides a scalable and efficient approach for adaptive multi-agent LLM systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 1 of its 2 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
1 of 2 references.
References · 1
- 2024Layerwise Recurrent Router for Mixture-of-Expertscited by 21
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Mixture-of-Agents Enhances Large Language Model Capabilities
- 2024SMoA: Improving Multi-agent Large Language Models with Sparse Mixture-of-Agents
- 2026Attention-MoA: Enhancing Mixture-of-Agents via Inter-Agent Semantic Attention and Deep Residual Synthesis
- 2025RMoA: Optimizing Mixture-of-Agents through Diversity Maximization and Residual Compensation
- 2026ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scaling
- 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
- 2026CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents
- 2025RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory