mlsanityData to 7 Oct 2026

Towards Generalized Routing: Model and Agent Orchestration for Adaptive and Efficient Inference

arXiv 2509.07571 ↗First posted 9 Sep 2025cs.MA cs.AIcited by 3

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The rapid advancement of large language models (LLMs) and domain-specific AI agents has greatly expanded the ecosystem of AI-powered services. User queries, however, are highly diverse and often span multiple domains and task types, resulting in a complex and heterogeneous landscape. This diversity presents a fundamental routing challenge: how to accurately direct each query to an appropriate execution unit while optimizing both performance and efficiency. To address this, we propose MoMA (Mixture of Models and Agents), a generalized routing framework that integrates both LLM and agent-based routing. Built upon a deep understanding of model and agent capabilities, MoMA effectively handles diverse queries through precise intent recognition and adaptive routing strategies, achieving an optimal balance between efficiency and cost. Specifically, we construct a detailed training dataset to profile the capabilities of various LLMs under different routing model structures, identifying the most suitable tasks for each LLM. During inference, queries are dynamically routed to the LLM with the best cost-performance efficiency. We also introduce an efficient agent selection strategy based on a context-aware state machine and dynamic masking. Experimental results demonstrate that the MoMA router offers superior cost-efficiency and scalability compared to existing approaches.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 24 of its 33 references that are in the map. First posted on arXiv 9 Sep 2025.

Builds on

24 of 33 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?2 Feb
  2. 2024Mixture-of-Agents Enhances Large Language Model Capabilities7 Jun

Other references · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  6. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  7. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  8. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  9. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  10. 2024Measuring short-form factuality in large language modelscited by 345
  11. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 154
  12. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 149
10 more
  1. 2023Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Modelscited by 141
  2. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  3. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 84
  4. 2024Mixture of Attention Spans: Optimizing LLM Inference Efficiency with Heterogeneous Sliding-Window Lengthscited by 44
  5. 2025The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giantscited by 37
  6. 2025Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routingcited by 32
  7. 2025Prompt-to-Leaderboardcited by 27
  8. 2025HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistantscited by 9
  9. 2023Extracting Protocol Format as State Machine via Controlled Static Loop Analysiscited by 2
  10. 2020Message Type Identification of Binary Network Protocols using Continuous Segment Similaritycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 35
  2. 2026ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inferencecited by 4
  3. 2026Holos: A Web-Scale LLM-Based Multi-Agent System for the Agentic Webcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2026Measured Joules, Learned Routes: Learning to Route for Energy-Efficient LLM Serving
  2. 2026Efficient and Interpretable Multi-Agent LLM Routing via Ant Colony Optimization
  3. 2026Agentic Routing: The Harness-Native Data Flywheel
  4. 2026Learning Agent Routing From Early Experience
  5. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey
  6. 2026RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents
  7. 2026Agent-as-a-Router: Agentic Model Routing for Coding Tasks
  8. 2025RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory