# Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation

arXiv: https://arxiv.org/abs/2105.14778 · HTML: https://mlsanity.com/paper/2105.14778/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement](https://mlsanity.com/paper/1802.06901/) (2018)
- [Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://mlsanity.com/paper/1907.12461/) (2019)
- [On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation](https://mlsanity.com/paper/2005.03642/) (2020)
- [Insertion Transformer: Flexible Sequence Generation via Insertion Operations](https://mlsanity.com/paper/1902.03249/) (2019)
- [Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://mlsanity.com/paper/1906.01081/) (2019)
- [Data-to-Text Generation with Content Selection and Planning](https://mlsanity.com/paper/1809.00582/) (2018)
- [Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data](https://mlsanity.com/paper/2010.05873/) (2020)
- [Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://mlsanity.com/paper/1904.03396/) (2019)
- [Long and Diverse Text Generation with Planning-based Hierarchical Variational Model](https://mlsanity.com/paper/1908.06605/) (2019)
- [Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation](https://mlsanity.com/paper/1910.08684/) (2019)
- [Fast Structured Decoding for Sequence Models](https://mlsanity.com/paper/1910.11555/) (2019)
- [Table-to-text Generation by Structure-aware Seq2seq Learning](https://mlsanity.com/paper/1711.09724/) (2017)
- [Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints](https://mlsanity.com/paper/2005.00969/) (2020)
- [Lexically Constrained Neural Machine Translation with Levenshtein Transformer](https://mlsanity.com/paper/2004.12681/) (2020)
- [Table-to-Text: Describing Table Region with Natural Language](https://mlsanity.com/paper/1805.11234/) (2018)
- [What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment](https://mlsanity.com/paper/1509.00838/) (2015)
- [Order-Planning Neural Text Generation From Structured Data](https://mlsanity.com/paper/1709.00155/) (2017)
- [Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation](https://mlsanity.com/paper/1908.03067/) (2019)
- [Describing a Knowledge Base](https://mlsanity.com/paper/1809.01797/) (2018)
- [Generating Descriptions from Structured Data Using a Bifocal Attention Mechanism and Gated Orthogonalization](https://mlsanity.com/paper/1804.07789/) (2018)

## Built on by (most cited first, 21 of 21)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)
- [Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods](https://mlsanity.com/paper/2203.05227/) (2022)
- [Zero-Resource Hallucination Prevention for Large Language Models](https://mlsanity.com/paper/2309.02654/) (2023)
- [Towards Verifiable Text Generation with Symbolic References](https://mlsanity.com/paper/2311.09188/) (2023)
- [Unifying Structured Data as Graph for Data-to-Text Pre-Training](https://mlsanity.com/paper/2401.01183/) (2024)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2310.18794/) (2023)
- [Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach](https://mlsanity.com/paper/2301.02071/) (2023)
- [Generating Faithful Text From a Knowledge Graph with Noisy Reference Text](https://mlsanity.com/paper/2308.06488/) (2023)
- [SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection](https://mlsanity.com/paper/2408.12748/) (2024)
- [Plan-then-Seam: Towards Efficient Table-to-Text Generation](https://mlsanity.com/paper/2302.05138/) (2023)
- [Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge Selection](https://mlsanity.com/paper/2308.15711/) (2023)
- [Measuring and Mitigating Hallucinations in Vision-Language Dataset Generation for Remote Sensing](https://mlsanity.com/paper/2501.14905/) (2025)
- [High Recall Data-to-text Generation with Progressive Edit](https://mlsanity.com/paper/2208.04558/) (2022)
- [Self-training from Self-memory in Data-to-text Generation](https://mlsanity.com/paper/2401.10567/) (2024)
- [Probing Omissions and Distortions in Transformer-based RDF-to-Text Models](https://mlsanity.com/paper/2409.16707/) (2024)
- [Is Our Chatbot Telling Lies? Assessing Correctness of an LLM-based Dutch Support Chatbot](https://mlsanity.com/paper/2411.00034/) (2024)
- [Data Transformation Strategies to Remove Heterogeneity](https://mlsanity.com/paper/2507.12677/) (2025)
- [From Tables to Quantified Statements: Evaluating LLM Inference Generation through Executable Verification](https://mlsanity.com/paper/2609.23966/) (2026)

## Abstract (arXiv)

Table-to-text generation refers to generating a descriptive text from a key-value table. Traditional autoregressive methods, though can generate text with high fluency, suffer from low coverage and poor faithfulness problems. To mitigate these problems, we propose a novel Skeleton-based two-stage method that combines both Autoregressive and Non-Autoregressive generations (SANA). Our approach includes: (1) skeleton generation with an autoregressive pointer network to select key tokens from the source table; (2) edit-based non-autoregressive generation model to produce texts via iterative insertion and deletion operations. By integrating hard constraints from the skeleton, the non-autoregressive model improves the generation's coverage over the source table and thus enhances its faithfulness. We conduct automatic and human evaluations on both WikiPerson and WikiBio datasets. Experimental results demonstrate that our method outperforms the previous state-of-the-art methods in both automatic and human evaluation, especially on coverage and faithfulness. In particular, we achieve PARENT-T recall of 99.47 in WikiPerson, improving over the existing best results by more than 10 points.