mlsanityData to 7 Oct 2026

Continual Model Routing in Evolving Model Hubs

arXiv 2605.28577 ↗First posted 27 May 2026cs.AI cs.LGcited by 1

On the map: Mixture-of-experts → Expert routing strategies

Abstract (arXiv)

AI model hubs provide access to a rapidly growing collection of powerful pre-trained models, enabling off-the-shelf mixture-of-experts systems with different routing strategies. However, this rapid growth poses two fundamental challenges: scaling model selection across thousands of experts and continually updating routing mechanisms as new models and tasks are introduced. In this paper, we formalise this setting as Continual Model Routing (CMR) and propose CMRBench, a new large-scale benchmark simulating realistic hub expansion and including over 2,000 candidate models. Finally, we introduce CARvE, a contrastive embedding approach for efficient continual model routing via checkpoint-based anchoring and structured replay. Extensive empirical results and ablations show that CARvE significantly outperforms zero-shot retrieval, fine-tuning, and adapter-merging baselines in model, family, and domain-level accuracy.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 35 of its 42 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

35 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity11 Jan

Other references · 34

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  6. 2016Learning without Forgettingcited by 2,465
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  8. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  9. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  10. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  11. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  12. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
22 more
  1. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  2. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  3. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  4. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 359
  5. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  6. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  7. 2023TaskBench: Benchmarking Large Language Models for Task Automationcited by 138
  8. 2023On the Tool Manipulation Capability of Open-source Large Language Modelscited by 122
  9. 2018Don't forget, there is more than forgetting: new metrics for Continual Learningcited by 113
  10. 2023AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Modelscited by 84
  11. 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
  12. 2024MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learningcited by 66
  13. 2025ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestrationcited by 44
  14. 2024LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversitycited by 42
  15. 2024API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMscited by 39
  16. 2024Routoo: Learning to Route to Large Language Models Effectivelycited by 34
  17. 2024Eagle: Efficient Training-Free Router for Multi-LLM Inferencecited by 34
  18. 2023Model Spider: Learning to Rank Pre-Trained Models Efficientlycited by 32
  19. 2025The Future of Continual Learning in the Era of Foundation Models: Three Key Directionscited by 16
  20. 2024Olympus: A Universal Task Router for Computer Vision Taskscited by 4
  21. 2025COLT: Enhancing Video Large Language Models with Continual Tool Usagecited by 2
  22. 2026UniRoute: Unified Routing Mixture-of-Experts for Modality-Adaptive Remote Sensing Change Detectioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness?cited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Adaptive Continual Model Merging via Manifold-Aware Expert Evolution
  2. 2026Stable Routing for Mixture-of-Experts in Class-Incremental Learning
  3. 2026Elbow-Based MoE Routing: A Training-Free Inference Time Plugin for Expert Selection
  4. 2026RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents
  5. 2025Routing-Based Continual Learning for Multimodal Large Language Models
  6. 2025LLaVA-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models
  7. 2026HERO-MoE: Historical Expert Routing with Scale-Preserving Fusion
  8. 2026A Persistent State for Auditable Mixture-of-Experts Routing