mlsanityData to 14 Oct 2026

LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks

arXiv 2507.05346 ↗First posted 7 Jul 2025cs.CL cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

The proliferation of fine-tuned language model experts for specific tasks and domains signals the need for efficient selection and combination methods. We propose LoRA-Augmented Generation (LAG) for leveraging large libraries of knowledge and task-specific LoRA adapters. LAG requires no additional training or access to data, and efficiently filters, retrieves, and applies experts on a per-token and layer basis. We evaluate LAG on various knowledge-intensive tasks, achieving superior performance over existing data-free methods. We explore scenarios where additional data is available, demonstrating LAG's compatibility with alternative solutions such as retrieval-augmented generation (RAG).

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

22 of 33 references.

References · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  10. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  11. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  12. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
10 more
  1. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  2. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  3. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 270
  4. 2023Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuningcited by 150
  5. 2022AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuningcited by 137
  6. 2024Towards Modular LLMs by Building and Reusing a Library of LoRAscited by 72
  7. 2022Multi-Head Adapter Routing for Cross-Task Generalizationcited by 33
  8. 2025Training Plug-n-Play Knowledge Modules with Deep Context Distillationcited by 24
  9. 2024KBLaM: Knowledge Base augmented Language Modelcited by 16
  10. 2024AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guaranteescited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generationcited by 16
  2. 2026Understanding LoRA as Knowledge Memory: An Empirical Analysiscited by 12
  3. 2026Rethinking LoRA Memory Through the Lens of KV Cache Compressioncited by 2
  4. 2026Parametric Memory Decoding for Zero-Shot Routing in LoRA-Based External Parametric Memorycited by 2
  5. 2025SEQR: Secure and Efficient QR-based LoRA Routingcited by 1
  6. 2026RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injectioncited by 1
  7. 2026Knowledge Pull Requests for Continual Document Authoringcited by 0
  8. 2026Towards In-Parameter Memory Augmentation for Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  3. 2024A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential
  4. 2024KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models
  5. 2024With Greater Text Comes Greater Necessity: Inference-Time Training Helps Long Text Generation
  6. 2025PRGB Benchmark: A Robust Placeholder-Assisted Algorithm for Benchmarking Retrieval-Augmented Generation
  7. 2023Active Retrieval Augmented Generation
  8. 2023Token-Level Adaptation of LoRA Adapters for Downstream Task Generalization