# Mixture of LoRA experts

Area: [Mixture-of-experts](https://mlsanity.com/moe/index.md). HTML: https://mlsanity.com/moe/mixture-of-lora-experts/

## Key facts

- Mixture of LoRA experts: 128 papers on arXiv since 2017 (data through 14 Oct 2026).
- 12 of its 128 mapped papers (9%) cite “Higher Layers Need More LoRA Experts” (2024, arXiv 2402.08562).
- 12 of its 128 mapped papers (9%) cite “Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters” (2024, arXiv 2403.11549).
- 7 new papers 17 Sep–14 Oct.

## Most cited here

- [Higher Layers Need More LoRA Experts](https://mlsanity.com/paper/2402.08562/) (2024), cited by 12 papers here
- [Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters](https://mlsanity.com/paper/2403.11549/) (2024), cited by 12 papers here
- [MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts](https://mlsanity.com/paper/2404.15159/) (2024), cited by 12 papers here
- [LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin](https://mlsanity.com/paper/2312.09979/) (2023), cited by 9 papers here
- [Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning](https://mlsanity.com/paper/2312.12379/) (2023), cited by 5 papers here
- [Theory on Mixture-of-Experts in Continual Learning](https://mlsanity.com/paper/2406.16437/) (2024), cited by 5 papers here
- [MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models](https://mlsanity.com/paper/2402.12851/) (2024), cited by 4 papers here
- [Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models](https://mlsanity.com/paper/2403.03432/) (2024), cited by 4 papers here
- [Mixture of Experts Meets Prompt-Based Continual Learning](https://mlsanity.com/paper/2405.14124/) (2024), cited by 4 papers here
- [SiRA: Sparse Mixture of Low Rank Adaptation](https://mlsanity.com/paper/2311.09179/) (2023), cited by 3 papers here

## New (17 Sep–14 Oct)

- [MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning](https://mlsanity.com/paper/2610.20238/) (2026-10-14)
- [How Many Ranks Does a Token Need? A Noise-Floor Squelch for Rank-One LoRA Experts](https://mlsanity.com/paper/2610.19033/) (2026-10-13)
- [Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs](https://mlsanity.com/paper/2610.18412/) (2026-10-13)
- [Monarch-MoE: Structured Experts for Multimodal Continual Instruction Tuning](https://mlsanity.com/paper/2610.13215/) (2026-10-09)
- [Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation](https://mlsanity.com/paper/2610.02123/) (2026-10-01)
- [S3: Spectral Null-Space Swap Makes Reasoning Models Efficient](https://mlsanity.com/paper/2609.37976/) (2026-09-29)
- [From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs](https://mlsanity.com/paper/2609.25655/) (2026-09-22)