mlsanityData to 14 Oct 2026

INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling

arXiv 2505.16303 ↗First posted 22 May 2025cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs, aiming to select the best-performing LLMs tailored to the domains of user queries, while managing computational resources. However, current routing approaches often face limitations in scalability when dealing with a large pool of specialized LLMs, or in their adaptability to extending model scope and evolving capability domains. To overcome those challenges, we propose InferenceDynamics, a flexible and scalable multi-dimensional routing framework by modeling the capability and knowledge of models. We operate it on our comprehensive dataset RouteMix, and demonstrate its effectiveness and generalizability in group-level routing using modern benchmarks including MMLU-Pro, GPQA, BigGenBench, and LiveBench, showcasing its ability to identify and leverage top-performing models for given tasks, leading to superior outcomes with efficient resource utilization. The broader adoption of Inference Dynamics can empower users to harness the full specialized potential of the LLM ecosystem, and our code will be made publicly available to encourage further research.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 35 of its 45 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

35 of 45 references.

References · 35

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  9. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  11. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  12. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
23 more
  1. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  2. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  3. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  4. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  5. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  6. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  7. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  8. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  9. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  10. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  11. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 326
  12. 2020Transformers as Soft Reasoners over Languagecited by 270
  13. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 263
  14. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?cited by 159
  15. 2020TLDR: Extreme Summarization of Scientific Documentscited by 136
  16. 2024NATURAL PLAN: Benchmarking LLMs on Natural Language Planningcited by 128
  17. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 93
  18. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  19. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  20. 2024Cost-Effective Online Multi-LLM Selection with Versatile Reward Modelscited by 39
  21. 2023Model Spider: Learning to Rank Pre-Trained Models Efficientlycited by 32
  22. 2025Prompt-to-Leaderboardcited by 30
  23. 2025Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routingcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025DiSRouter: Distributed Self-Routing for LLM Selectionscited by 5
  2. 2026ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inferencecited by 5
  3. 2025XToM: Exploring the Multilingual Theory of Mind for Large Language Modelscited by 2
  4. 2025Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Frameworkcited by 2
  5. 2026ConsRoute:Consistency-Aware Adaptive Query Routing for Cloud-Edge-Device Large Language Modelscited by 1
  6. 2026VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrievalcited by 1
  7. 2026Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routingcited by 0
  8. 2026Routing Should Pay for Itself: Sparse Supervision for Economical LLM Routingcited by 0