mlsanityData to 7 Oct 2026

Focused Attention Improves Document-Grounded Generation

arXiv 2104.12714 ↗First posted 26 Apr 2021cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Document grounded generation is the task of using the information provided in a document to improve text generation. This work focuses on two different document grounded generation tasks: Wikipedia Update Generation task and Dialogue response generation. Our work introduces two novel adaptations of large scale pre-trained encoder-decoder models focusing on building context driven representation of the document and enabling specific attention to the information in the document. Additionally, we provide a stronger BART baseline for these tasks. Our proposed techniques outperform existing methods on both automated (at least 48% increase in BLEU-4 points) and human evaluation for closeness to reference and relevance to the document. Furthermore, we perform comprehensive manual inspection of the generated output and categorize errors to provide insights into future directions in modeling these tasks.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 38 of its 58 references that are in the map. First posted on arXiv 26 Apr 2021.

Builds on

38 of 58 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  7. 2018Hierarchical Neural Story Generationcited by 1,182
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  10. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  11. 2018CoQA: A Conversational Question Answering Challengecited by 674
  12. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
26 more
  1. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  2. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
  3. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
  4. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  5. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  6. 2017Challenges in Data-to-Document Generationcited by 226
  7. 2015A Neural Conversational Modelcited by 195
  8. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  9. 2016Visual Storytellingcited by 176
  10. 2017A Knowledge-Grounded Neural Conversation Modelcited by 136
  11. 2015Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systemscited by 134
  12. 2020Knowledge-Grounded Dialogue Generation with Pre-trained Language Modelscited by 107
  13. 2018A Dataset for Document Grounded Conversationscited by 104
  14. 2017Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generationcited by 61
  15. 2017Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generationcited by 59
  16. 2018Polite Dialogue Generation Without Parallel Datacited by 57
  17. 2020Exploring Controllable Text Generation Techniquescited by 55
  18. 2020Low-Resource Knowledge-Grounded Dialogue Generationcited by 53
  19. 2016Personalized Machine Translation: Preserving Original Author Traitscited by 47
  20. 2019Multimodal Abstractive Summarization for How2 Videoscited by 41
  21. 2020Text Editing by Commandcited by 33
  22. 2015What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignmentcited by 31
  23. 2018Augmenting Neural Response Generation with Context-Aware Topical Attentioncited by 27
  24. 2020Pretrained Language Models for Dialogue Generation with Multiple Input Sourcescited by 14
  25. 2018MS-UEdin Submission to the WMT2018 APE Shared Task: Dual-Source Transformer for Automatic Post-Editingcited by 13
  26. 2019Towards Content Transfer through Grounded Text Generationcited by 9

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 11; 2023: 6; 2024: 4; 2025: 1.

24 papers.

Sort:

By area:

  1. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  2. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  3. 2021Evaluating Attribution in Dialogue Systems: The BEGIN Benchmarkcited by 75
  4. 2022OpenCQA: Open-ended Question Answering with Chartscited by 60
  5. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  6. 2021Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identitycited by 22
  7. 2022Multi-Stage Prompting for Knowledgeable Dialogue Generationcited by 22
  8. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  9. 2023Pointwise Mutual Information Based Metric and Decoding Strategy for Faithful Generation in Document Grounded Dialogscited by 11
  10. 2022Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Taskscited by 8
  11. 2025Evaluating Evaluation Metrics -- The Mirage of Hallucination Detectioncited by 6
  12. 2023KnowPrefix-Tuning: A Two-Stage Prefix-Tuning Framework for Knowledge-Grounded Dialogue Generationcited by 5
12 more
  1. 2022Learning as Conversation: Dialogue Systems Reinforced for Information Acquisitioncited by 4
  2. 2022CorefDiffs: Co-referential and Differential Knowledge Flow in Document Grounded Conversationscited by 4
  3. 2022Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injectioncited by 3
  4. 2022Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Modelcited by 3
  5. 2024Transformers Utilization in Chart Understanding: A Review of Recent Advances & Future Trendscited by 3
  6. 2022Building Knowledge-Grounded Dialogue Systems with Graph-Based Semantic Modelingcited by 2
  7. 2022Dynamically Retrieving Knowledge via Query Generation for Informative Dialogue Generationcited by 2
  8. 2023Contextual Knowledge Learning For Dialogue Generationcited by 1
  9. 2023Graph vs. Sequence: An Empirical Study on Knowledge Forms for Knowledge-Grounded Dialoguecited by 1
  10. 2023Blending Reward Functions via Few Expert Demonstrations for Faithful and Accurate Knowledge-Grounded Dialogue Generationcited by 0
  11. 2024UniRQR: A Unified Model for Retrieval Decision, Query, and Response Generation in Internet-Based Knowledge Dialogue Systemscited by 0
  12. 2024Policy-driven Knowledge Selection and Response Generation for Document-grounded Dialoguecited by 0