mlsanityData to 7 Oct 2026

KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation

arXiv 2403.14950 ↗First posted 22 Mar 2024cs.CL cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Parameter-efficient finetuning (PEFT) is a key technique for adapting large language models (LLMs) to downstream tasks. In this paper, we study leveraging knowledge graph embeddings to improve the effectiveness of PEFT. We propose a knowledgeable adaptation method called KnowLA. It inserts an adaptation layer into an LLM to integrate the embeddings of entities appearing in the input text. The adaptation layer is trained in combination with LoRA on instruction data. Experiments on six benchmarks with two popular LLMs and three knowledge graphs demonstrate the effectiveness and robustness of KnowLA. We show that modelname can help activate the relevant parameterized knowledge in an LLM to answer a question without changing its parameters or input prompts.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 38 of its 57 references that are in the map. First posted on arXiv 22 Mar 2024.

Builds on

38 of 57 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
26 more
  1. 2020GLU Variants Improve Transformercited by 1,724
  2. 2019Root Mean Square Layer Normalizationcited by 1,425
  3. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  4. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  5. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  6. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  7. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 893
  8. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  9. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  10. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  11. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  12. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 303
  13. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 296
  14. 2019Knowledge Enhanced Contextual Word Representationscited by 294
  15. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
  16. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  17. 2023PMET: Precise Model Editing in a Transformercited by 170
  18. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  19. 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
  20. 2020KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoningcited by 106
  21. 2020CoLAKE: Contextualized Language and Knowledge Embeddingcited by 86
  22. 2019SenseBERT: Driving Some Sense into BERTcited by 73
  23. 2022Lifelong Embedding Learning and Transfer for Growing Knowledge Graphscited by 28
  24. 2022Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignmentcited by 24
  25. 2021KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphscited by 19
  26. 2022Enhancing Document-level Relation Extraction by Entity Knowledge Injectioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 1.

10 papers.

  1. 2024DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literaturecited by 50
  2. 2025Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunitiescited by 30
  3. 2025Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answeringcited by 22
  4. 2025Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Modelscited by 11
  5. 2025Aligning Knowledge Graphs and Language Models for Factual Accuracycited by 4
  6. 2024LLM+KG@VLDB'24 Workshop Summarycited by 3
  7. 2024Gene-Metabolite Association Prediction with Interactive Knowledge Transfer Enhanced Graph for Metabolite Productioncited by 1
  8. 2025Knowledge Fusion via Bidirectional Information Aggregationcited by 1
  9. 2025Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMscited by 0
  10. 2026KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Modelscited by 0