mlsanityData to 7 Oct 2026

MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency

arXiv 2606.03014 ↗First posted 2 Jun 2026cs.LG cs.ARcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Mixture-of-Agents (MoA) systems improve reasoning accuracy by routing each query to multiple expert LLMs and aggregating their outputs. Efficiently executing this workload on limited GPU resources has bottlenecks. Skill-based routing creates skewed expert demand, and combining instruction-tuned LLMs with long-reasoning models results in extreme variability in generation lengths. Consequently, traditional scheduling strategies suffer from significant GPU idling and throughput collapse due to load imbalances. We present MOSAIC, a scheduling framework to accelerate MoA workloads. First, we formulate an Integer Linear Program (ILP) based scheduler that jointly optimizes expert placement and per-worker prompt assignment from offline-profiled costs, replicating reasoning experts across workers while pinning lightweight ones. Second, MOSAIC uses confidence-aware adaptive aggregation, leveraging inter-expert agreement to bypass the heavy final aggregator LLM for consensus queries. In our 4-GPU system, MOSAIC achieves up to 2.5x expert-stage, 4.23x aggregator-stage and 1.7~2.3x end-to-end speedups over the baseline scheduler, while matching accuracy within 0.1pp.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 28 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

14 of 28 references.

References · 14

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  4. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  5. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  7. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  8. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  9. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 403
  10. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  11. 2024EXAONE 3.5: Series of Large Language Models for Real-world Use Casescited by 50
  12. 2024MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Servingcited by 49
2 more
  1. 2026Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoningcited by 17
  2. 2025Slm-mux: Orchestrating small language models for reasoningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Adaptive GPU Resource Allocation for Multi-Agent Collaborative Reasoning in Serverless Environments
  2. 2025Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills
  3. 2025Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony
  4. 2026SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters
  5. 2025Justitia: Fair and Efficient Scheduling of Task-parallel LLM Agents with Selective Pampering
  6. 2026ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scaling
  7. 2026Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling
  8. 2026CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving