# Denoising Pre-Training and Data Augmentation Strategies for Enhanced RDF Verbalization with Transformers

arXiv: https://arxiv.org/abs/2012.00571 · HTML: https://mlsanity.com/paper/2012.00571/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 10 of 10)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Data Augmentation Approaches in Natural Language Processing: A Survey](https://mlsanity.com/paper/2110.01852/) (2021)
- [Retrieve, Caption, Generate: Visual Grounding for Enhancing Commonsense in Text Generation Models](https://mlsanity.com/paper/2109.03892/) (2021)
- [SAPPHIRE: Approaches for Enhanced Concept-to-Text Generation](https://mlsanity.com/paper/2108.06643/) (2021)
- [Leveraging Open Information Extraction for More Robust Domain Transfer of Event Trigger Detection](https://mlsanity.com/paper/2305.14163/) (2023)
- [OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs](https://mlsanity.com/paper/2503.11858/) (2025)
- [Neural Data-to-Text Generation Based on Small Datasets: Comparing the Added Value of Two Semi-Supervised Learning Approaches on Top of a Large Language Model](https://mlsanity.com/paper/2207.06839/) (2022)
- [Investigating Masking-based Data Generation in Language Models](https://mlsanity.com/paper/2307.00008/) (2023)
- [Probing Omissions and Distortions in Transformer-based RDF-to-Text Models](https://mlsanity.com/paper/2409.16707/) (2024)
- [TailNLG: A Multilingual Benchmark Addressing Verbalization of Long-Tail Entities](https://mlsanity.com/paper/2603.27768/) (2026)

## Abstract (arXiv)

The task of verbalization of RDF triples has known a growth in popularity due to the rising ubiquity of Knowledge Bases (KBs). The formalism of RDF triples is a simple and efficient way to store facts at a large scale. However, its abstract representation makes it difficult for humans to interpret. For this purpose, the WebNLG challenge aims at promoting automated RDF-to-text generation. We propose to leverage pre-trainings from augmented data with the Transformer model using a data augmentation strategy. Our experiment results show a minimum relative increases of 3.73%, 126.05% and 88.16% in BLEU score for seen categories, unseen entities and unseen categories respectively over the standard training.