mlsanityData to 7 Oct 2026

Augmented Large Language Models with Parametric Knowledge Guiding

arXiv 2305.04757 ↗First posted 8 May 2023cs.CL cs.AI cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have significantly advanced natural language processing (NLP) with their impressive language understanding and generation capabilities. However, their performance may be suboptimal for domain-specific tasks that require specialized knowledge due to limited exposure to the related data. Additionally, the lack of transparency of most state-of-the-art (SOTA) LLMs, which can only be accessed via APIs, impedes further fine-tuning with domain custom data. Moreover, providing private data to the LLMs' owner leads to data privacy problems. To address these challenges, we propose the novel Parametric Knowledge Guiding (PKG) framework, which equips LLMs with a knowledge-guiding module to access relevant knowledge without altering the LLMs' parameters. Our PKG is based on open-source "white-box" language models, allowing offline memory of any knowledge that LLMs require. We demonstrate that our PKG framework can enhance the performance of "black-box" LLMs on a range of domain knowledge-intensive tasks that require factual (+7.9%), tabular (+11.9%), medical (+3.0%), and multimodal (+8.1%) knowledge.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 8 May 2023.

Builds on

46 of 59 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  12. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
34 more
  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  6. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  8. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  9. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  10. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  11. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  12. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  13. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  14. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  15. 2023Instruction Tuning with GPT-4cited by 676
  16. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  17. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  18. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  19. 2023Augmented Language Models: a Surveycited by 398
  20. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  21. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  22. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  23. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  24. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  25. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  26. 2019End-to-End Open-Domain Question Answering with BERTserinicited by 152
  27. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  28. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
  29. 2021Open Domain Question Answering over Tables via Dense Retrievalcited by 74
  30. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  31. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  32. 2021Fool Me Twice: Entailment from Wikipedia Gamificationcited by 50
  33. 2023ChatGPT may Pass the Bar Exam soon, but has a Long Way to Go for the LexGLUE benchmarkcited by 33
  34. 2023AI and the FCI: Can ChatGPT Project an Understanding of Introductory Physics?cited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 15; 2025: 7; 2026: 3.

37 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 176
  4. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  5. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 106
  6. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  7. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  8. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
  9. 2024Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?cited by 47
  10. 2023Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Modelscited by 46
  11. 2023CITB: A Benchmark for Continual Instruction Tuningcited by 32
  12. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
25 more
  1. 2024C-RAG: Certified Generation Risks for Retrieval-Augmented Language Modelscited by 27
  2. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  3. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  4. 2024KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Modelscited by 20
  5. 2024WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledgecited by 16
  6. 2023ATLANTIC: Structure-Aware Retrieval-Augmented Language Model for Interdisciplinary Sciencecited by 15
  7. 2023ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?cited by 15
  8. 2025Cancer-Myth: Evaluating Large Language Models on Patient Questions with False Presuppositionscited by 15
  9. 2024Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leapscited by 12
  10. 2024Chaos with Keywords: Exposing Large Language Models Sycophantic Hallucination to Misleading Keywords and Evaluating Defense Strategiescited by 10
  11. 2024RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generationcited by 10
  12. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
  13. 2026MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillationcited by 6
  14. 2025ECoRAG: Evidentiality-guided Compression for Long Context RAGcited by 4
  15. 2023SCALE: Synergized Collaboration of Asymmetric Language Translation Enginescited by 3
  16. 2024FaiMA: Feature-aware In-context Learning for Multi-domain Aspect-based Sentiment Analysiscited by 1
  17. 2024A Self-guided Multimodal Approach to Enhancing Graph Representation Learning for Alzheimer's Diseasescited by 1
  18. 2024Towards a Unified Paradigm: Integrating Recommendation Systems as a New Language in Large Modelscited by 1
  19. 2025Guided Navigation in Knowledge-Dense Environments: Structured Semantic Exploration with Guidance Graphscited by 1
  20. 2024Fine-grained Stateful Knowledge Exploration: Effective and Efficient Graph Retrieval with Large Language Modelscited by 0
  21. 2024Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Modelscited by 0
  22. 2025CCSK:Cognitive Convection of Self-Knowledge Based Retrieval Augmentation for Large Language Modelscited by 0
  23. 2025Xinyu AI Search: Enhanced Relevance and Comprehensive Results with Rich Answer Presentationscited by 0
  24. 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistencycited by 0
  25. 2026Attribution Techniques for Mitigating Hallucinated Information in RAG Systems: A Surveycited by 0