# Toward Super Agent System with Hybrid AI Routers

arXiv: https://arxiv.org/abs/2504.10519 · HTML: https://mlsanity.com/paper/2504.10519/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 11 Apr 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 22 of 22)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing](https://mlsanity.com/paper/2404.14618/) (2024)
- [Scaling Large Language Model-based Multi-Agent Collaboration](https://mlsanity.com/paper/2406.07155/) (2024)
- [RouterBench: A Benchmark for Multi-LLM Routing System](https://mlsanity.com/paper/2403.12031/) (2024)
- [GraphRouter: A Graph-based Router for LLM Selections](https://mlsanity.com/paper/2410.03834/) (2024)
- [PlanGenLLMs: A Modern Survey of LLM Planning Capabilities](https://mlsanity.com/paper/2502.11221/) (2025)
- [ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning](https://mlsanity.com/paper/2308.13724/) (2023)
- [TensorOpera Router: A Multi-Model Router for Efficient LLM Inference](https://mlsanity.com/paper/2408.12320/) (2024)
- [LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning](https://mlsanity.com/paper/2401.00125/) (2023)
- [TinyAgent: Function Calling at the Edge](https://mlsanity.com/paper/2409.00608/) (2024)
- [Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomies](https://mlsanity.com/paper/2309.13063/) (2023)
- [Large Language Models for Intent-Driven Session Recommendations](https://mlsanity.com/paper/2312.07552/) (2023)
- [Intent Detection in the Age of LLMs](https://mlsanity.com/paper/2410.01627/) (2024)
- [Online Cascade Learning for Efficient Inference over Streams](https://mlsanity.com/paper/2402.04513/) (2024)
- [User Intent Recognition and Satisfaction with Large Language Models: A User Study with ChatGPT](https://mlsanity.com/paper/2402.02136/) (2024)
- [Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection](https://mlsanity.com/paper/2402.17256/) (2024)
- [Complex LLM Planning via Automated Heuristics Discovery](https://mlsanity.com/paper/2502.19295/) (2025)
- [LANID: LLM-assisted New Intent Discovery](https://mlsanity.com/paper/2503.23740/) (2025)
- [Resource-efficient Inference with Foundation Model Programs](https://mlsanity.com/paper/2504.07247/) (2025)

## Built on by (most cited first, 6 of 6)

- [A Survey on Collaborative Mechanisms Between Large and Small Language Models](https://mlsanity.com/paper/2505.07460/) (2025)
- [ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models](https://mlsanity.com/paper/2510.27256/) (2025)
- [PAAC: Privacy-Aware Agentic Device-Cloud Collaboration](https://mlsanity.com/paper/2605.08646/) (2026)
- [Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2509.04459/) (2025)
- [BusMA: A Bus Communication Substrate for Multi-Agent Systems](https://mlsanity.com/paper/2609.15054/) (2026)
- [Intelligent Design 4.0: Paradigm Evolution Toward the Agentic AI Era](https://mlsanity.com/paper/2506.09755/) (2025)

## Abstract (arXiv)

AI Agents powered by Large Language Models are transforming the world through enormous applications. A super agent has the potential to fulfill diverse user needs, such as summarization, coding, and research, by accurately understanding user intent and leveraging the appropriate tools to solve tasks. However, to make such an agent viable for real-world deployment and accessible at scale, significant optimizations are required to ensure high efficiency and low cost. This position paper presents a design of the Super Agent System powered by the hybrid AI routers. Upon receiving a user prompt, the system first detects the intent of the user, then routes the request to specialized task agents with the necessary tools or automatically generates agentic workflows. In practice, most applications directly serve as AI assistants on edge devices such as phones and robots. As different language models vary in capability and cloud-based models often entail high computational costs, latency, and privacy concerns, we then explore the hybrid mode where the router dynamically selects between local and cloud models based on task complexity. Finally, we introduce the blueprint of an on-device super agent enhanced with cloud. With advances in multi-modality models and edge hardware, we envision that most computations can be handled locally, with cloud collaboration only as needed. Such architecture paves the way for super agents to be seamlessly integrated into everyday life in the near future.