# Faithful Low-Resource Data-to-Text Generation through Cycle Training

arXiv: https://arxiv.org/abs/2305.14793 · HTML: https://mlsanity.com/paper/2305.14793/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [The E2E Dataset: New Challenges For End-to-End Generation](https://mlsanity.com/paper/1706.09254/) (2017)
- [TAPEX: Table Pre-training via Learning a Neural SQL Executor](https://mlsanity.com/paper/2107.07653/) (2021)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [DART: Open-Domain Structured Data Record to Text Generation](https://mlsanity.com/paper/2007.02871/) (2020)
- [Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://mlsanity.com/paper/1906.01081/) (2019)
- [Learning Dense Correspondence via 3D-guided Cycle Consistency](https://mlsanity.com/paper/1604.05383/) (2016)
- [TableFormer: Robust Transformer Modeling for Table-Text Encoding](https://mlsanity.com/paper/2203.00274/) (2022)
- [CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training](https://mlsanity.com/paper/2006.04702/) (2020)
- [WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections](https://mlsanity.com/paper/2012.14919/) (2020)
- [Few-Shot Table-to-Text Generation with Prototype Memory](https://mlsanity.com/paper/2108.12516/) (2021)
- [ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models](https://mlsanity.com/paper/2210.04325/) (2022)

## Built on by (most cited first, 7 of 7)

- [LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagation](https://mlsanity.com/paper/2408.15533/) (2024)
- [Shallow Synthesis of Knowledge in GPT-Generated Texts: A Case Study in Automatic Related Work Composition](https://mlsanity.com/paper/2402.12255/) (2024)
- [Construction of Paired Knowledge Graph-Text Datasets Informed by Cyclic Evaluation](https://mlsanity.com/paper/2309.11669/) (2023)
- [Think While You Write: Hypothesis Verification Promotes Faithful Knowledge-to-Text Generation](https://mlsanity.com/paper/2311.09467/) (2023)
- [Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction](https://mlsanity.com/paper/2401.10189/) (2024)
- [Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model](https://mlsanity.com/paper/2310.13106/) (2023)
- [Self-training from Self-memory in Data-to-text Generation](https://mlsanity.com/paper/2401.10567/) (2024)

## Abstract (arXiv)

Methods to generate text from structured data have advanced significantly in recent years, primarily due to fine-tuning of pre-trained language models on large datasets. However, such models can fail to produce output faithful to the input data, particularly on out-of-domain data. Sufficient annotated data is often not available for specific domains, leading us to seek an unsupervised approach to improve the faithfulness of output text. Since the problem is fundamentally one of consistency between the representations of the structured data and text, we evaluate the effectiveness of cycle training in this work. Cycle training uses two models which are inverses of each other: one that generates text from structured data, and one which generates the structured data from natural language text. We show that cycle training, when initialized with a small amount of supervised data (100 samples in our case), achieves nearly the same performance as fully supervised approaches for the data-to-text generation task on the WebNLG, E2E, WTQ, and WSQL datasets. We perform extensive empirical analysis with automated evaluation metrics and a newly designed human evaluation schema to reveal different cycle training strategies' effectiveness of reducing various types of generation errors. Our code is publicly available at https://github.com/Edillower/CycleNLG.