mlsanityData to 7 Oct 2026

Attention Is Indeed All You Need: Semantically Attention-Guided Decoding for Data-to-Text NLG

arXiv 2109.07043 ↗First posted 15 Sep 2021cs.CL cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Ever since neural models were adopted in data-to-text language generation, they have invariably been reliant on extrinsic components to improve their semantic accuracy, because the models normally do not exhibit the ability to generate text that reliably mentions all of the information provided in the input. In this paper, we propose a novel decoding method that extracts interpretable information from encoder-decoder models' cross-attention, and uses it to infer which attributes are mentioned in the generated text, which is subsequently used to rescore beam hypotheses. Using this decoding method with T5 and BART, we show on three datasets its ability to dramatically reduce semantic errors in the generated outputs, while maintaining their state-of-the-art quality.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

28 of 37 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  9. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  10. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  11. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  12. 2015Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systemscited by 134
16 more
  1. 2018Learning to Write with Cooperative Discriminatorscited by 119
  2. 2016Modeling Coverage for Neural Machine Translationcited by 110
  3. 2020Text-to-Text Pre-Training for Data-to-Text Taskscited by 104
  4. 2020Few-shot Natural Language Generation for Task-Oriented Dialogcited by 89
  5. 2020Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelitycited by 35
  6. 2018Generating More Interesting Responses in Neural Conversation Models with Distributional Constraintscited by 33
  7. 2019Pragmatically Informative Text Generationcited by 32
  8. 2019ViGGO: A Video Game Corpus for Data-To-Text Generation in Open-Domain Conversationcited by 31
  9. 2018Findings of the E2E NLG Challengecited by 28
  10. 2016Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Stringscited by 27
  11. 2020Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondencecited by 25
  12. 2018A Deep Ensemble Model with Slot Alignment for Sequence-to-Sequence Natural Language Generationcited by 20
  13. 2015Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Rerankingcited by 17
  14. 2019A Good Sample is Hard to Find: Noise Injection Sampling and Self-Training for Neural Language Generation Modelscited by 17
  15. 2017Learning to generate one-sentence biographies from Wikidatacited by 13
  16. 2018Char2char Generation with Reranking for the E2E NLG Challengecited by 5

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 3; 2024: 2; 2025: 2.

11 papers.

  1. 2023Controlling Personality Style in Dialogue with Zero-Shot Prompt-Based Learningcited by 14
  2. 2025Towards Transparent AI: A Survey on Explainable Large Language Modelscited by 7
  3. 2021Jurassic is (almost) All You Need: Few-Shot Meaning-to-Text Generation for Open-Domain Dialoguecited by 6
  4. 2021Athena 2.0: Contextualized Dialogue Management for an Alexa Prize SocialBotcited by 4
  5. 2023Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Rankingcited by 4
  6. 2024A Systematic Review of Data-to-Text NLGcited by 4
  7. 2025Towards Transparent AI: A Survey on Explainable Language Modelscited by 3
  8. 2023You Can Generate It Again: Data-to-Text Generation with Verification and Correction Promptingcited by 2
  9. 2024Source Identification in Abstractive Summarizationcited by 2
  10. 2022Comparing Computational Architectures for Automated Journalismcited by 0
  11. 2022Reranking Overgenerated Responses for End-to-End Task-Oriented Dialogue Systemscited by 0