mlsanityData to 7 Oct 2026

Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuning

arXiv 2410.10360 ↗First posted 14 Oct 2024cs.CL cs.IRcited by 14

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) offers an effective solution to the issues faced by Large Language Models (LLMs) in hallucination generation and knowledge obsolescence by incorporating externally retrieved knowledge. However, existing methods lack effective control mechanisms for integrating internal and external knowledge. Inspired by human cognitive processes, we propose Parenting, a novel framework that decouples, identifies, and purposefully optimizes parameter subspaces related to adherence and robustness. Specifically, Parenting utilizes a key parameter mining method that combines forward and backward propagation signals to localize subspaces representing different capabilities. Then, Parenting employs a type-tailored tuning strategy, applying specific and appropriate optimizations to different subspaces, aiming to achieve a balanced enhancement of both adherence and robustness. Extensive experiments on various datasets and models validate the effectiveness and generalizability of our method.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 38 of its 57 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

38 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training31 May
  2. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence16 Apr
  3. 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models22 Feb
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2023Qwen Technical Reportcited by 3,273
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2020GLU Variants Improve Transformercited by 1,724
  12. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
22 more
  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  2. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  3. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  4. 2019Importance Estimation for Neural Network Pruningcited by 531
  5. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  6. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  7. 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 337
  8. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  9. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  10. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  11. 2022Prompting GPT-3 To Be Reliablecited by 258
  12. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  13. 2022Large Language Models with Controllable Working Memorycited by 169
  14. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  15. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  16. 2024Universal Neurons in GPT2 Language Modelscited by 100
  17. 2024Not All Layers of LLMs Are Necessary During Inferencecited by 95
  18. 2022PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importancecited by 79
  19. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  20. 2024IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neuronscited by 36
  21. 2024TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systemscited by 28
  22. 2023HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responsescited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 6.

14 papers.

Sort:

By area:

  1. 2024RAGraph: A General Retrieval-Augmented Graph Learning Frameworkcited by 24
  2. 2025Recurrent Knowledge Identification and Fusion for Language Model Continual Learningcited by 14
  3. 2026StackPlanner: A Centralized Hierarchical Multi-Agent System with Task-Experience Memory Managementcited by 13
  4. 2025GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigationcited by 11
  5. 2026FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learningcited by 7
  6. 2025GeoEdit: Geometric Knowledge Editing for Large Language Modelscited by 6
  7. 2026Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generationcited by 6
  8. 2025AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learningcited by 4
  9. 2025ADEPT: Continual Pretraining via Adaptive Expansion and Dynamic Decoupled Tuningcited by 4
  10. 2025HetaRAG: Hybrid Deep Retrieval-Augmented Generation across Heterogeneous Data Storescited by 3
  11. 2025DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Searchcited by 2
  12. 2026ScaffoldAgent: Utility-Guided Dynamic Outline Optimization for Open-Ended Deep Researchcited by 2
2 more
  1. 2026Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Modelscited by 0
  2. 2026Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge
  2. 2024Fine-Grained Guidance for Retrievers: Leveraging LLMs' Feedback in Retrieval-Augmented Generation
  3. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models
  4. 2025Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation
  5. 2023"Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation
  6. 2025Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG
  7. 2025Towards Lighter and Robust Evaluation for Retrieval Augmented Generation
  8. 2025MultiRAG: A Knowledge-guided Framework for Mitigating Hallucination in Multi-source Retrieval Augmented Generation