mlsanityData to 14 Oct 2026

IR3DE: A Linear Router for Large Language Models

arXiv 2606.06098 ↗First posted 4 Jun 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LLMs. With the ever-growing list of available LLMs, inference routers are being proposed to select the most appropriate LLM for each prompt. However, existing routing methods either optimize cost across weak-to-strong generalist LLMs or require substantial training to support domain-expertise routing. In this paper, we propose IR3DE, a Ridge Regression-based Router for Domain Experts that provides cheap and fast routing decisions for each prompt. We evaluate IR3DE in two Causal Language Modeling (CLM) settings where the tasks are next-token prediction for all domains, and one reasoning setting where each domain has its own distinct reasoning task. Despite being a linear router, IR3DE achieves performance comparable to the other baselines in both CLM settings, and surpassing them in the reasoning setting, with a normalized performance of 98.4%. Moreover, IR3DE enables the addition or removal of new domain experts without requiring the router to be retrained from scratch, allowing a dynamic set of LLMs to be served with minimal disruption to the router itself. Our code is available at: github.com/gensyn-ai/IR3DE.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

30 of 38 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  7. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  8. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  9. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  10. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  11. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  12. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 190
18 more
  1. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 165
  2. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  3. 2025Universal Model Routing for Efficient LLM Inferencecited by 86
  4. 2024A Unified Approach to Routing and Cascading for LLMscited by 80
  5. 2025IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theorycited by 40
  6. 2025MergeBench: A Benchmark for Merging Domain-Specialized LLMscited by 33
  7. 2021Towards Model Agnostic Federated Learning Using Knowledge Distillationcited by 31
  8. 2025ICL-Router: In-Context Learned Model Representations for LLM Routingcited by 28
  9. 2022M2D2: A Massively Multi-domain Language Modeling Datasetcited by 26
  10. 2022Coresets for Vertical Federated Learning: Regularized Linear Regression and K-Means Clusteringcited by 14
  11. 2026R2-Router: A New Paradigm for LLM Routing with Reasoningcited by 14
  12. 2024Accelerating Heterogeneous Federated Learning with Closed-form Classifierscited by 10
  13. 2026Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarioscited by 10
  14. 2024MoDEM: Mixture of Domain Expert Modelscited by 9
  15. 2025DiSRouter: Distributed Self-Routing for LLM Selectionscited by 5
  16. 2025HDEE: Heterogeneous Domain Expert Ensemblecited by 4
  17. 2025HierRouter: Coordinated Routing of Specialized Large Language Models via Reinforcement Learningcited by 3
  18. 2026Training-Free Dynamic Upcycling of Expert Language Modelscited by 1

Built on by

Not cited yet.