# Neural Pipeline for Zero-Shot Data-to-Text Generation

arXiv: https://arxiv.org/abs/2203.16279 · HTML: https://mlsanity.com/paper/2203.16279/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 113 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 30 of 53)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [The E2E Dataset: New Challenges For End-to-End Generation](https://mlsanity.com/paper/1706.09254/) (2017)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [Higher-order Coreference Resolution with Coarse-to-fine Inference](https://mlsanity.com/paper/1804.05392/) (2018)
- [DART: Open-Domain Structured Data Record to Text Generation](https://mlsanity.com/paper/2007.02871/) (2020)
- [Investigating Pretrained Language Models for Graph-to-Text Generation](https://mlsanity.com/paper/2007.08426/) (2020)
- [Data-to-Text Generation with Content Selection and Planning](https://mlsanity.com/paper/1809.00582/) (2018)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Text-to-Text Pre-Training for Data-to-Text Tasks](https://mlsanity.com/paper/2005.10433/) (2020)
- [Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge](https://mlsanity.com/paper/1901.07931/) (2019)
- [Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://mlsanity.com/paper/1904.03396/) (2019)
- [Semantic Noise Matters for Neural Natural Language Generation](https://mlsanity.com/paper/1911.03905/) (2019)
- [Long and Diverse Text Generation with Planning-based Hierarchical Variational Model](https://mlsanity.com/paper/1908.06605/) (2019)
- [Learning Neural Templates for Text Generation](https://mlsanity.com/paper/1808.10122/) (2018)
- [JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs](https://mlsanity.com/paper/2106.10502/) (2021)
- [Few-Shot NLG with Pre-Trained Language Model](https://mlsanity.com/paper/1904.09521/) (2019)
- [Plan-then-Generate: Controlled Data-to-Text Generation via Planning](https://mlsanity.com/paper/2108.13740/) (2021)
- [Neural data-to-text generation: A comparison between pipeline and end-to-end architectures](https://mlsanity.com/paper/1908.09022/) (2019)
- [Changing the World by Changing the Data](https://mlsanity.com/paper/2105.13947/) (2021)

## Built on by (most cited first, 19 of 19)

- [HallE-Control: Controlling Object Hallucination in Large Multimodal Models](https://mlsanity.com/paper/2310.01779/) (2023)
- [Language-Guided Music Recommendation for Video via Prompt Analogies](https://mlsanity.com/paper/2306.09327/) (2023)
- [Evaluating Generative Models for Graph-to-Text Generation](https://mlsanity.com/paper/2307.14712/) (2023)
- [ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models](https://mlsanity.com/paper/2210.04325/) (2022)
- [MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation](https://mlsanity.com/paper/2212.08607/) (2022)
- [Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models](https://mlsanity.com/paper/2210.07373/) (2022)
- [PixT3: Pixel-based Table-To-Text Generation](https://mlsanity.com/paper/2311.09808/) (2023)
- [A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation](https://mlsanity.com/paper/2402.01495/) (2024)
- [Narrating Causal Graphs with Large Language Models](https://mlsanity.com/paper/2403.07118/) (2024)
- [PAGED: A Benchmark for Procedural Graphs Extraction from Documents](https://mlsanity.com/paper/2408.03630/) (2024)
- [Ontology-Free General-Domain Knowledge Graph-to-Text Generation Dataset Synthesis using Large Language Model](https://mlsanity.com/paper/2409.07088/) (2024)
- [You Can Generate It Again: Data-to-Text Generation with Verification and Correction Prompting](https://mlsanity.com/paper/2306.15933/) (2023)
- [Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning](https://mlsanity.com/paper/2308.05317/) (2023)
- [Automatic Logical Forms improve fidelity in Table-to-Text generation](https://mlsanity.com/paper/2310.17279/) (2023)
- [ASPIRO: Any-shot Structured Parsing-error-Induced ReprOmpting for Consistent Data-to-Text Generation](https://mlsanity.com/paper/2310.17877/) (2023)
- [Leveraging Large Language Models for Building Interpretable Rule-Based Data-to-Text Systems](https://mlsanity.com/paper/2502.20609/) (2025)
- [JaccDiv: A Metric and Benchmark for Quantifying Diversity of Generated Marketing Text in the Music Industry](https://mlsanity.com/paper/2504.20849/) (2025)
- [LLM Agents Implement an NLG System from Scratch: Building Interpretable Rule-Based RDF-to-Text Generators](https://mlsanity.com/paper/2512.18360/) (2025)
- [RAG-NAROK: Retrieval-Aware Knowledge Corpus Poisoning in RAG with Source-specific Refutation](https://mlsanity.com/paper/2609.25469/) (2026)

## Abstract (arXiv)

In data-to-text (D2T) generation, training on in-domain data leads to overfitting to the data representation and repeating training data noise. We examine how to avoid finetuning pretrained language models (PLMs) on D2T generation datasets while still taking advantage of surface realization capabilities of PLMs. Inspired by pipeline approaches, we propose to generate text by transforming single-item descriptions with a sequence of modules trained on general-domain text-based operations: ordering, aggregation, and paragraph compression. We train PLMs for performing these operations on a synthetic corpus WikiFluent which we build from English Wikipedia. Our experiments on two major triple-to-text datasets -- WebNLG and E2E -- show that our approach enables D2T generation from RDF triples in zero-shot settings.