mlsanityData to 7 Oct 2026

What Has Been Enhanced in my Knowledge-Enhanced Language Model?

arXiv 2202.00964 ↗First posted 2 Feb 2022cs.CL cs.IT cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Pretrained language models (LMs) do not capture factual knowledge very well. This has led to the development of a number of knowledge integration (KI) methods which aim to incorporate external knowledge into pretrained LMs. Even though KI methods show some performance gains over vanilla LMs, the inner-workings of these methods are not well-understood. For instance, it is unclear how and what kind of knowledge is effectively integrated into these models and if such integration may lead to catastrophic forgetting of already learned knowledge. This paper revisits the KI process in these models with an information-theoretic view and shows that KI can be interpreted using a graph convolution operation. We propose a probe model called textitGraph Convolution Simulator (GCS) for interpreting knowledge-enhanced LMs and exposing what kind of knowledge is integrated into these models. We conduct experiments to verify that our GCS can indeed be used to correctly interpret the KI process, and we use it to analyze two well-known knowledge-enhanced LMs: ERNIE and K-Adapter, and find that only a small amount of factual knowledge is integrated in them. We stratify knowledge in terms of various relation types and find that ERNIE and K-Adapter integrate different kinds of knowledge to different extent. Our analysis also shows that simply increasing the size of the KI corpus may not lead to better KI; fundamental advances may be needed.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 40 of its 57 references that are in the map. First posted on arXiv 2 Feb 2022.

Builds on

40 of 57 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2017Graph Attention Networkscited by 8,854
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  10. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  11. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  12. 2018MINE: Mutual Information Neural Estimationcited by 680
28 more
  1. 2019Designing and Interpreting Probes with Control Taskscited by 608
  2. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  3. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  4. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  5. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  6. 2019KG-BERT: BERT for Knowledge Graph Completioncited by 323
  7. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  8. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 303
  9. 2019Knowledge Enhanced Contextual Word Representationscited by 294
  10. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  11. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  12. 2020Information-Theoretic Probing for Linguistic Structurecited by 204
  13. 2019Universal Invariant and Equivariant Graph Neural Networkscited by 182
  14. 2019On the equivalence between graph isomorphism testing and function approximation with GNNscited by 172
  15. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  16. 2020Interpreting Graph Neural Networks for NLP With Differentiable Edge Maskingcited by 152
  17. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  18. 2018Attention-based Graph Neural Network for Semi-supervised Learningcited by 100
  19. 2018Ultra-Fine Entity Typingcited by 87
  20. 2013Discrete Signal Processing on Graphs: Frequency Analysiscited by 72
  21. 2020Temporal Reasoning on Implicit Events from Distant Supervisioncited by 58
  22. 2021Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mappingcited by 47
  23. 2019Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embeddingcited by 38
  24. 2019Smooth function approximation by deep neural networks with general activation functionscited by 29
  25. 2020Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehensioncited by 17
  26. 2020Understanding Graph Neural Networks from Graph Signal Denoising Perspectivescited by 11
  27. 2021Bird's Eye: Probing for Linguistic Graph Structures with a Simple Information-Theoretic Approachcited by 9
  28. 2015Handling Class Imbalance in Link Prediction using Learning to Rank Techniquescited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2024Combining Knowledge Graphs and Large Language Modelscited by 16
  2. 2023Revisiting the Knowledge Injection Frameworkscited by 4