mlsanityData to 14 Oct 2026

JevSoup: System-One Routing for Training-Free LoRA Composition

arXiv 2609.30922 ↗First posted 25 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Building adaptable AI systems requires effective coordination of specialized capabilities across diverse tasks. Low-rank adaptation (LoRA) enables modular expertise, but existing routing approaches may require auxiliary data, additional training, or autoregressive decoding. We propose JevSoup, a training-free framework separating System One expert routing from System Two execution. Using only the input and expert descriptions, Jev selects two experts through structured probabilities. JevSoup retains the leading expert's update, projects the second onto the orthogonal complement of the first update's row space, and combines them with equal weights. Across 14 PorTAL tasks and three Qwen3 scales, JepSoup achieves absolute gains of up to 1.19% in task-macro and 1.21% in sample-micro accuracy over the strongest evaluated external baselines. Our code is available at https://github.com/Leowang980/JevSoup.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

15 of 18 references.

References · 15

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  4. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  5. 2019Orthogonal Gradient Descent for Continual Learningcited by 369
  6. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  7. 2024Mixture of LoRA Expertscited by 181
  8. 2023AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Modelscited by 84
  9. 2024Towards Modular LLMs by Building and Reusing a Library of LoRAscited by 72
  10. 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
  11. 2026τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledgecited by 39
  12. 2026Effective LoRA Adapter Routing using Task Representationscited by 6
3 more
  1. 2025Adaptive Minds: Empowering Agents with LoRA-as-Toolscited by 5
  2. 2025HiLoRA: Adaptive Hierarchical LoRA Routing for Training-Free Domain Generalizationcited by 4
  3. 2025LoRA on the Go: Instance-level Dynamic LoRA Selection and Mergingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Score the Update, Not the Token: Descent-Aligned Routing for Combinatorial LoRA Experts
  2. 2026MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
  3. 2026Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA
  4. 2026Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs
  5. 2025Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
  6. 2026MoE2-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation
  7. 2025TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
  8. 2025LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts