# XF2T: Cross-lingual Fact-to-Text Generation for Low-Resource Languages

arXiv: https://arxiv.org/abs/2209.11252 · HTML: https://mlsanity.com/paper/2209.11252/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 22 Sep 2022.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [The E2E Dataset: New Challenges For End-to-End Generation](https://mlsanity.com/paper/1706.09254/) (2017)
- [XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation](https://mlsanity.com/paper/2004.01401/) (2020)
- [MuRIL: Multilingual Representations for Indian Languages](https://mlsanity.com/paper/2103.10730/) (2021)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://mlsanity.com/paper/1906.01081/) (2019)
- [Investigating Pretrained Language Models for Graph-to-Text Generation](https://mlsanity.com/paper/2007.08426/) (2020)
- [NCLS: Neural Cross-Lingual Summarization](https://mlsanity.com/paper/1909.00156/) (2019)
- [Cross-Lingual Natural Language Generation via Pre-Training](https://mlsanity.com/paper/1909.10481/) (2019)
- [Entity Linking in 100 Languages](https://mlsanity.com/paper/2011.02690/) (2020)
- [MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs](https://mlsanity.com/paper/2104.08692/) (2021)
- [What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment](https://mlsanity.com/paper/1509.00838/) (2015)
- [ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generation](https://mlsanity.com/paper/2106.01597/) (2021)
- [WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections](https://mlsanity.com/paper/2012.14919/) (2020)
- [Multi-domain Neural Network Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1603.01232/) (2016)
- [Neural Wikipedian: Generating Textual Summaries from Knowledge Base Triples](https://mlsanity.com/paper/1711.00155/) (2017)
- [Generating Descriptions from Structured Data Using a Bifocal Attention Mechanism and Gated Orthogonalization](https://mlsanity.com/paper/1804.07789/) (2018)
- [Stage-wise Fine-tuning for Graph-to-Text Generation](https://mlsanity.com/paper/2105.08021/) (2021)
- [Partially-Aligned Data-to-Text Generation with Distant Supervision](https://mlsanity.com/paper/2010.01268/) (2020)
- [Two Birds, One Stone: A Simple, Unified Model for Text Generation from Structured and Unstructured Data](https://mlsanity.com/paper/1909.10158/) (2019)
- [Multilingual Fact Linking](https://mlsanity.com/paper/2109.14364/) (2021)
- [XAlign: Cross-lingual Fact-to-Text Alignment and Generation for Low-Resource Languages](https://mlsanity.com/paper/2202.00291/) (2022)

## Built on by (most cited first, 1 of 1)

- [XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languages](https://mlsanity.com/paper/2303.12308/) (2023)

## Abstract (arXiv)

Multiple business scenarios require an automated generation of descriptive human-readable text from structured input data. Hence, fact-to-text generation systems have been developed for various downstream tasks like generating soccer reports, weather and financial reports, medical reports, person biographies, etc. Unfortunately, previous work on fact-to-text (F2T) generation has focused primarily on English mainly due to the high availability of relevant datasets. Only recently, the problem of cross-lingual fact-to-text (XF2T) was proposed for generation across multiple languages alongwith a dataset, XALIGN for eight languages. However, there has been no rigorous work on the actual XF2T generation problem. We extend XALIGN dataset with annotated data for four more languages: Punjabi, Malayalam, Assamese and Oriya. We conduct an extensive study using popular Transformer-based text generation models on our extended multi-lingual dataset, which we call XALIGNV2. Further, we investigate the performance of different text generation strategies: multiple variations of pretraining, fact-aware embeddings and structure-aware input encoding. Our extensive experiments show that a multi-lingual mT5 model which uses fact-aware embeddings with structure-aware input encoding leads to best results on average across the twelve languages. We make our code, dataset and model publicly available, and hope that this will help advance further research in this critical area.