mlsanityData to 7 Oct 2026

Lexical Knowledge Internalization for Neural Dialog Generation

arXiv 2205.01941 ↗First posted 4 May 2022cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models. Instead of further conditioning the knowledge-grounded dialog (KGD) models on externally retrieved knowledge, we seek to integrate knowledge about each input token internally into the model's parameters. To tackle the challenge due to the large scale of lexical knowledge, we adopt the contrastive learning approach and create an effective token-level lexical knowledge retriever that requires only weak supervision mined from Wikipedia. We demonstrate the effectiveness and general applicability of our approach on various datasets and diversified model structures.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 4 May 2022.

Builds on

38 of 56 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  10. 2019How Can We Know What Language Models Know?cited by 953
  11. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  12. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
26 more
  1. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  2. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  3. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  4. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 303
  5. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
  6. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  7. 2017A Knowledge-Grounded Neural Conversation Modelcited by 136
  8. 2015Neural Responding Machine for Short-Text Conversationcited by 124
  9. 2020Knowledge-Grounded Dialogue Generation with Pre-trained Language Modelscited by 107
  10. 2015A Neural Network Approach to Context-Sensitive Generation of Conversational Responsescited by 98
  11. 2020Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervisioncited by 93
  12. 2019Proactive Human-Machine Conversation with Explicit Conversation Goalscited by 87
  13. 2020Sequential Latent Knowledge Selection for Knowledge-Grounded Dialoguecited by 82
  14. 2017Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generationcited by 59
  15. 2019Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphscited by 59
  16. 2018Towards Exploiting Background Knowledge for Building Conversation Systemscited by 57
  17. 2020Low-Resource Knowledge-Grounded Dialogue Generationcited by 53
  18. 2019Incremental Transformer with Deliberation Decoder for Document Grounded Conversationscited by 35
  19. 2019Conversing by Reading: Contentful Neural Conversation with On-demand Machine Readingcited by 32
  20. 2020OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contextscited by 27
  21. 2021Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapterscited by 23
  22. 2018Retrieval-Enhanced Adversarial Training for Neural Response Generationcited by 22
  23. 2017Flexible End-to-End Dialogue System for Knowledge Grounded Conversationcited by 18
  24. 2020Syntactic Structure Distillation Pretraining For Bidirectional Encoderscited by 18
  25. 2018Extending Neural Generative Conversational Model using External Knowledge Sourcescited by 16
  26. 2020Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generationcited by 14

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Contrastive Learning Reduces Hallucination in Conversationscited by 39