# Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach

arXiv: https://arxiv.org/abs/2301.02071 · HTML: https://mlsanity.com/paper/2301.02071/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 5 Jan 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [Data-to-Text Generation with Content Selection and Planning](https://mlsanity.com/paper/1809.00582/) (2018)
- [Text-to-Text Pre-Training for Data-to-Text Tasks](https://mlsanity.com/paper/2005.10433/) (2020)
- [Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model](https://mlsanity.com/paper/2010.09142/) (2020)
- [Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation](https://mlsanity.com/paper/1910.08684/) (2019)
- [Table-to-text Generation by Structure-aware Seq2seq Learning](https://mlsanity.com/paper/1711.09724/) (2017)
- [Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints](https://mlsanity.com/paper/2005.00969/) (2020)
- [What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment](https://mlsanity.com/paper/1509.00838/) (2015)
- [TSQA: Tabular Scenario Based Question Answering](https://mlsanity.com/paper/2101.11429/) (2021)
- [Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation](https://mlsanity.com/paper/2105.14778/) (2021)
- [Table-to-Text Generation with Effective Hierarchical Encoder on Three Dimensions (Row, Column and Time)](https://mlsanity.com/paper/1909.02304/) (2019)
- [Pre-Translation for Neural Machine Translation](https://mlsanity.com/paper/1610.05243/) (2016)
- [Rewriter-Evaluator Architecture for Neural Machine Translation](https://mlsanity.com/paper/2012.05414/) (2020)

## Built on by (most cited first, 4 of 4)

- [GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking](https://mlsanity.com/paper/2305.15066/) (2023)
- [Think While You Write: Hypothesis Verification Promotes Faithful Knowledge-to-Text Generation](https://mlsanity.com/paper/2311.09467/) (2023)
- [Control Large Language Models via Divide and Conquer](https://mlsanity.com/paper/2410.04628/) (2024)
- [Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo](https://mlsanity.com/paper/2404.04103/) (2024)

## Abstract (arXiv)

Although remarkable progress on the neural table-to-text methods has been made, the generalization issues hinder the applicability of these models due to the limited source tables. Large-scale pretrained language models sound like a promising solution to tackle such issues. However, how to effectively bridge the gap between the structured table and the text input by fully leveraging table information to fuel the pretrained model is still not well explored. Besides, another challenge of integrating the deliberation mechanism into the text-to-text pretrained model for solving the table-to-text task remains seldom studied. In this paper, to implement the table-to-text generation with pretrained language model, we propose a table structure understanding and text deliberating approach, namely TASD. Specifically, we devise a three-layered multi-head attention network to realize the table-structure-aware text generation model with the help of the pretrained language model. Furthermore, a multi-pass decoder framework is adopted to enhance the capability of polishing generated text for table descriptions. The empirical studies, as well as human evaluation, on two public datasets, validate that our approach can generate faithful and fluent descriptive texts for different types of tables.