mlsanityData to 14 Oct 2026

vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models

arXiv 2603.04444 ↗First posted 23 Feb 2026cs.NI cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) diversify across modalities, capabilities, and cost profiles, the problem of intelligent request routing: selecting the right model for each query at inference time, has become a critical systems challenge. We present vLLM Semantic Router, a signal-driven decision routing framework for Mixture-of-Modality (MoM) model deployments. The architecture follows two complementary Shannon-inspired views. In the information-theoretic regime, signal extraction reduces the entropy of "which model?" by distilling routing-relevant information from raw queries. In the Boolean-algebraic regime, the decision engine composes functionally complete routing policies from signal conditions. The central innovation is composable signal orchestration: thirteen heterogeneous signal types, spanning sub-millisecond heuristics and neural classifiers for semantics, safety, and modality, are composed through configurable Boolean decision rules into deployment-specific routing policies, so that fundamentally different scenarios (multi-cloud enterprise, privacy-regulated, cost-optimized) are expressed as different configurations over the same architecture. Matched decisions drive semantic model routing via thirteen selection algorithms, while per-decision plugin chains enforce safety constraints including a three-stage HaluGate hallucination detection pipeline and a lightweight episodic memory system with ReflectionGate for personalized multi-turn context. A typed neural-symbolic DSL specifies these routing policies and compiles them to multiple deployment targets, enabling configuration-first adaptation without code changes. Together, these components show that composable signal orchestration enables a single framework to serve diverse deployment scenarios with differentiated cost, privacy, and safety policies.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 19 of its 42 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

19 of 42 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  7. 2024Mixtral of Expertscited by 1,823
  8. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  10. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  11. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
7 more
  1. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  2. 2022Matryoshka Representation Learningcited by 282
  3. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  4. 2023AutoMix: Automatically Mixing Language Modelscited by 125
  5. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 88
  6. 2025Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learningcited by 72
  7. 2025GMTRouter: Personalized LLM Router over Multi-turn User Interactionscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routingcited by 6
  2. 2026Calibrate-Then-Delegate: Safety Monitoring with Risk and Budget Guarantees via Model Cascadescited by 3
  3. 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Projectcited by 1
  4. 2026SEAR: Schema-Based Evaluation and Routing for LLM Gatewayscited by 1
  5. 2026TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AIcited by 1
  6. 2026Handoff Curves: Initial Selection, Later Handoff, and Supervision Targets in Multi-turn LLM Routingcited by 0