mlsanityData to 14 Oct 2026

MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora

arXiv 2507.09924 ↗First posted 14 Jul 2025cs.IR cs.AI cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Continually updating model-based indexes in generative retrieval with new documents remains challenging, as full retraining is computationally expensive and impractical under resource constraints. We propose MixLoRA-DSI, a novel framework that combines an expandable mixture of Low-Rank Adaptation experts with a layer-wise out-of-distribution (OOD)-driven expansion strategy. Instead of allocating new experts for each new corpus, our proposed expansion strategy enables sublinear parameter growth by selectively introducing new experts only when significant number of OOD documents are detected. Experiments on NQ320k and MS MARCO Passage demonstrate that MixLoRA-DSI outperforms full-model update baselines, with minimal parameter overhead and substantially lower training costs.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 36 of its 61 references that are in the map. First posted on arXiv 14 Jul 2025.

Builds on

36 of 61 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2016Layer Normalizationcited by 5,669
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  11. 2017Billion-scale similarity search with GPUscited by 2,497
  12. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
24 more
  1. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  2. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  3. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  4. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  5. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  6. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  7. 2018Task-Free Continual Learningcited by 221
  8. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  9. 2020A Neural Dirichlet Process Mixture Model for Task-Free Continual Learningcited by 134
  10. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 127
  11. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 113
  12. 2023Learning to Tokenize for Generative Retrievalcited by 68
  13. 2022A Closer Look at Rehearsal-Free Continual Learningcited by 54
  14. 2023How Does Generative Retrieval Scale to Millions of Passages?cited by 48
  15. 2023Learning to Rank in Generative Retrievalcited by 46
  16. 2023Scalable and Effective Generative Information Retrievalcited by 39
  17. 2022Ultron: An Ultimate Retriever on Corpus with a Model-based Indexercited by 31
  18. 2023Continual Learning for Generative Retrieval over Dynamic Corporacited by 25
  19. 2024Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decodingcited by 19
  20. 2023Generative Retrieval via Term Set Generationcited by 13
  21. 2024CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Taskscited by 10
  22. 2023L^2R: Lifelong Learning for First-stage Retrieval with Backward-Compatible Representationscited by 8
  23. 2023Exploring the Practicality of Generative Retrieval on Dynamic Corporacited by 6
  24. 2024ASI++: Towards Distributionally Balanced End-to-End Generative Retrievalcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026A Parametric Memory Head for Continual Generative Retrievalcited by 1
  2. 2026Routing Is Not Enough: Diagnosing Intra-Adapter Subspace Contention in MoE+LoRA Fine-Tuningcited by 1
  3. 2026ICICLE: Expanding Retrieval with In-Context Documentscited by 0
  4. 2026GUIDE: Guiding Internal Evidence with Language Instructionscited by 0