mlsanityData to 14 Oct 2026

Parametric Memory Decoding for Zero-Shot Routing in LoRA-Based External Parametric Memory

arXiv 2607.04118 ↗First posted 5 Jul 2026cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

With the rise of parametric memory, LoRA-based External Parametric Memory (EPM) has emerged as a modular solution, but existing routing methods often introduce additional training, deployment, and maintenance overhead. This raises a natural question: can a LoRA-based EPM bank be routed without maintaining an additional routing component? However, existing zero-shot LoRA routing methods still face two problems under the EPM setting: (1) their evaluations are scattered across different task settings rather than organized around EPM access, and (2) their routing signals lack a unified perspective to guide systematic improvement. To address these problems, we organize PMD-Bench, covering document-level, domain-level knowledge, and task-skill, and propose Parametric Memory Decoding (PMD), the first framework designed to systematically improve zero-shot LoRA routing by reframing it as decoding activations over external parametric memory. Based on PMD, we further instantiate PMDRouter, which scores each LoRA by its response magnitude from a single base-model prefill. Experiments on PMD-Bench show that PMDRouter achieves the strongest internal-signal performance across multiple zero-shot routing settings. These results demonstrate the feasibility of zero-shot LoRA routing and suggest that PMD can serve as a general framework for improving zero-shot routing methods. Sources: Github (https://anonymous.4open.science/r/Parametric-Memory-Decoding-872A/)

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 59 of its 73 references that are in the map. First posted on arXiv 5 Jul 2026.

Builds on

59 of 73 references.

References · 59

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  5. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  6. 2024Mixture of LoRA Expertscited by 181
  7. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 150
  8. 2024MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Expertscited by 147
  9. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  10. 2024MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuningcited by 94
  11. 2024WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Modelscited by 92
  12. 2024Towards Modular LLMs by Building and Reusing a Library of LoRAscited by 72
38 more
  1. 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
  2. 2024Learning to Route Among Specialized Experts for Zero-Shot Generalizationcited by 65
  3. 2024LoRA Soups: Merging LoRAs for Practical Skill Composition Taskscited by 52
  4. 2024X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Designcited by 42
  5. 2018Memory-based Parameter Adaptationcited by 35
  6. 2022Multi-Head Adapter Routing for Cross-Task Generalizationcited by 33
  7. 2025MLP Memory: A Retriever-Pretrained Memory for Large Language Modelscited by 20
  8. 2025Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Mergingcited by 19
  9. 2024LoRA-Switch: Boosting the Efficiency of Dynamic LLM Adapters via System-Algorithm Co-designcited by 18
  10. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queriescited by 16
  11. 2025Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Modelscited by 16
  12. 2026Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Modelscited by 15
  13. 2024Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learningcited by 14
  14. 2025LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Expertscited by 13
  15. 2024Exploring Training on Heterogeneous Data with Mixture of Low-rank Adapterscited by 12
  16. 2024Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Modelscited by 11
  17. 2025DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanismcited by 11
  18. 2025Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMscited by 10
  19. 2025Hy-Facial: Hybrid Feature Extraction by Dimensionality Reduction Methods for Enhanced Facial Expression Classificationcited by 10
  20. 2024Evaluating the External and Parametric Knowledge Fusion of Large Language Modelscited by 9
  21. 2025LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Taskscited by 8
  22. 2025MemLoRA: Distilling Expert Adapters for On-Device Memory Systemscited by 7
  23. 2026Memory Intelligence Agentcited by 7
  24. 2025SpectR: Dynamically Composing LM Experts with Spectral Routingcited by 6
  25. 2026Effective LoRA Adapter Routing using Task Representationscited by 6
  26. 2025Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Expertscited by 5
  27. 2025AutoLoRA: Automatic LoRA Retrieval and Fine-Grained Gated Fusion for Text-to-Image Generationcited by 5
  28. 2026FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosurescited by 5
  29. 2026FineState-Bench: Benchmarking State-Conditioned Grounding for Fine-grained GUI State Settingcited by 5
  30. 2026TextAlign: Preference Alignment for Text Rendering with Hierarchical Rewardscited by 5
  31. 2025HiLoRA: Adaptive Hierarchical LoRA Routing for Training-Free Domain Generalizationcited by 4
  32. 2026CoMoL: Efficient Mixture of LoRA Experts via Dynamic Core Space Mergingcited by 4
  33. 2026ServImage: An Image Generation and Editing Benchmark from Real-world Commercial Imaging Servicescited by 4
  34. 2026Zero-Parameter Geometric Gating for Temporally Stable Low-Altitude UAV Video Semantic Segmentationcited by 4
  35. 2025Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generationcited by 3
  36. 2024Review of Zero-Shot and Few-Shot AI Algorithms in The Medical Domaincited by 2
  37. 2024DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Modelscited by 2
  38. 2024Beyond Adapter Retrieval: Latent Geometry-Preserving Composition via Sparse Task Projectioncited by 2

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generationcited by 2
  2. 2026TransMem: Transforming Hidden States into Memory for Large Language Modelscited by 1