A Neural Knowledge Language Model
Not on the map. Why?
Abstract (arXiv)
Current language models have a significant limitation in the ability to encode and decode factual knowledge. This is mainly because they acquire such knowledge from statistical co-occurrences although most of the knowledge words are rarely observed. In this paper, we propose a Neural Knowledge Language Model (NKLM) which combines symbolic knowledge provided by the knowledge graph with the RNN language model. By predicting whether the word to generate has an underlying fact or not, the model can generate such knowledge-related words by copying from the description of the predicted fact. In experiments, we show that the NKLM significantly improves the performance while generating a much smaller number of unknown words.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Aug 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 6; 2022: 5; 2023: 0; 2024: 1; 2025: 1.
13 papers.
Sort:
- 2021Improving language models by retrieving from trillions of tokenscited by 1,278
- 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
- 2022A Review on Language Models as Knowledge Basescited by 156
- 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
- 2022Relational Memory Augmented Language Modelscited by 24
- 2021Reasoning Over Virtual Knowledge Bases With Open Predicate Relationscited by 18
- 2025Pretraining with hierarchical memories: separating long-tail and common knowledgecited by 13
- 2022Innovations in Neural Data-to-text Generation: A Surveycited by 10
- 2021WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Webcited by 5
- 2021Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRUcited by 3
- 2021TopNet: Learning from Neural Topic Model to Generate Long Storiescited by 2
- 2021Contextual Text Embeddings for Twicited by 1
1 more
- 2025Pretraining with hierarchical memories: separating long-tail and common knowledge29 Sep 2025
- 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems11 Jan 2024
- 2022Innovations in Neural Data-to-text Generation: A Survey25 Jul 2022
- 2022Factuality Enhanced Language Models for Open-Ended Text Generation9 Jun 2022
- 2022SubGraph Networks based Entity Alignment for Cross-lingual Knowledge Graph7 May 2022
- 2022A Review on Language Models as Knowledge Bases12 Apr 2022
- 2022Relational Memory Augmented Language Models24 Jan 2022
- 2021TopNet: Learning from Neural Topic Model to Generate Long Stories14 Dec 2021
- 2021Improving language models by retrieving from trillions of tokens8 Dec 2021
- 2021Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRU17 Jun 2021
- 2021Contextual Text Embeddings for Twi29 Mar 2021
- 2021WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web18 Feb 2021
1 more
- 2021Improving language models by retrieving from trillions of tokenscited by 1,278
- 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
- 2022A Review on Language Models as Knowledge Basescited by 156
- 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
- 2022Relational Memory Augmented Language Modelscited by 24
- 2021Reasoning Over Virtual Knowledge Bases With Open Predicate Relationscited by 18
- 2025Pretraining with hierarchical memories: separating long-tail and common knowledgecited by 13
- 2022Innovations in Neural Data-to-text Generation: A Surveycited by 10
- 2021WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Webcited by 5
- 2021Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRUcited by 3
- 2021TopNet: Learning from Neural Topic Model to Generate Long Storiescited by 2
- 2021Contextual Text Embeddings for Twicited by 1
12 most cited of 13 All 13 (JSON)
- 2025Pretraining with hierarchical memories: separating long-tail and common knowledge29 Sep 2025
- 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems11 Jan 2024
- 2022Innovations in Neural Data-to-text Generation: A Survey25 Jul 2022
- 2022Factuality Enhanced Language Models for Open-Ended Text Generation9 Jun 2022
- 2022SubGraph Networks based Entity Alignment for Cross-lingual Knowledge Graph7 May 2022
- 2022A Review on Language Models as Knowledge Bases12 Apr 2022
- 2022Relational Memory Augmented Language Models24 Jan 2022
- 2021TopNet: Learning from Neural Topic Model to Generate Long Stories14 Dec 2021
- 2021Improving language models by retrieving from trillions of tokens8 Dec 2021
- 2021Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRU17 Jun 2021
- 2021Contextual Text Embeddings for Twi29 Mar 2021
- 2021WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web18 Feb 2021
12 newest of 13 All 13 (JSON)