# Self-training from Self-memory in Data-to-text Generation

arXiv: https://arxiv.org/abs/2401.10567 · HTML: https://mlsanity.com/paper/2401.10567/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [The E2E Dataset: New Challenges For End-to-End Generation](https://mlsanity.com/paper/1706.09254/) (2017)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [Extractive Summarization as Text Matching](https://mlsanity.com/paper/2004.08795/) (2020)
- [BRIO: Bringing Order to Abstractive Summarization](https://mlsanity.com/paper/2203.16804/) (2022)
- [DART: Open-Domain Structured Data Record to Text Generation](https://mlsanity.com/paper/2007.02871/) (2020)
- [Plan-then-Generate: Controlled Data-to-Text Generation via Planning](https://mlsanity.com/paper/2108.13740/) (2021)
- [Neural data-to-text generation: A comparison between pipeline and end-to-end architectures](https://mlsanity.com/paper/1908.09022/) (2019)
- [Data-to-text Generation with Entity Modeling](https://mlsanity.com/paper/1906.03221/) (2019)
- [CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training](https://mlsanity.com/paper/2006.04702/) (2020)
- [Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity](https://mlsanity.com/paper/2004.06577/) (2020)
- [Data-to-text Generation with Macro Planning](https://mlsanity.com/paper/2102.02723/) (2021)
- [Pragmatically Informative Text Generation](https://mlsanity.com/paper/1904.01301/) (2019)
- [Findings of the E2E NLG Challenge](https://mlsanity.com/paper/1810.01170/) (2018)
- [Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation](https://mlsanity.com/paper/2105.14778/) (2021)
- [Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation](https://mlsanity.com/paper/1908.03067/) (2019)
- [A Hierarchical Model for Data-to-Text Generation](https://mlsanity.com/paper/1912.10011/) (2019)
- [Improving Compositional Generalization with Self-Training for Data-to-Text Generation](https://mlsanity.com/paper/2110.08467/) (2021)
- [Describing a Knowledge Base](https://mlsanity.com/paper/1809.01797/) (2018)
- [A Good Sample is Hard to Find: Noise Injection Sampling and Self-Training for Neural Language Generation Models](https://mlsanity.com/paper/1911.03373/) (2019)
- [Crowd-sourcing NLG Data: Pictures Elicit Better Data](https://mlsanity.com/paper/1608.00339/) (2016)
- [Template-free Data-to-Text Generation of Finnish Sports News](https://mlsanity.com/paper/1910.01863/) (2019)
- [Faithful Low-Resource Data-to-Text Generation through Cycle Training](https://mlsanity.com/paper/2305.14793/) (2023)
- [Copy mechanism and tailored training for character-based data-to-text generation](https://mlsanity.com/paper/1904.11838/) (2019)
- [LOGEN: Few-shot Logical Knowledge-Conditioned Text Generation with Self-training](https://mlsanity.com/paper/2112.01404/) (2021)

## Abstract (arXiv)

This paper introduces a novel training model, self-training from self-memory (STSM) in data-to-text generation (DTG), allowing the model to self-train on subsets, including self-memory as outputs inferred directly from the trained models and/or the new data. The quality of self-memory is validated by two models, data-to-text (D2T) and text-to-data (T2D), by two pre-defined conditions: (1) the appearance of all source values in the outputs of the D2T model and (2) the ability to convert back to source data in the outputs in the T2D model. We utilize a greedy algorithm to generate shorter D2T outputs if they contain all source values. Subsequently, we use the T2D model to confirm that these outputs can capture input relationships by demonstrating their capacity to convert text back into data. With 30% of the dataset, we can train the D2T model with a competitive performance compared to full training in the same setup. We experiment with our model on two datasets, E2E NLG and DART. STSM offers the D2T model a generalization capability from its subset memory while reducing training data volume. Ultimately, we anticipate that this paper will contribute to continual learning solutions that adapt to new training data, incorporating it as a form of self-memory in DTG tasks. The curated dataset is publicly available at: https://github.com/hoangthangta/STSM.