# Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models

arXiv: https://arxiv.org/abs/2106.01623 · HTML: https://mlsanity.com/paper/2106.01623/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 3 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019)
- [Text Generation from Knowledge Graphs with Graph Transformers](https://mlsanity.com/paper/1904.02342/) (2019)
- [Investigating Pretrained Language Models for Graph-to-Text Generation](https://mlsanity.com/paper/2007.08426/) (2020)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2105.10311/) (2021)
- [Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://mlsanity.com/paper/1904.03396/) (2019)
- [Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling](https://mlsanity.com/paper/1906.07241/) (2019)
- [Few-Shot NLG with Pre-Trained Language Model](https://mlsanity.com/paper/1904.09521/) (2019)
- [CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training](https://mlsanity.com/paper/2006.04702/) (2020)
- [Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs](https://mlsanity.com/paper/2001.11003/) (2020)
- [Structural Information Preserving for Graph-to-Text Generation](https://mlsanity.com/paper/2102.06749/) (2021)
- [TextBox: A Unified, Modularized, and Extensible Framework for Text Generation](https://mlsanity.com/paper/2101.02046/) (2021)

## Built on by (most cited first, 29 of 29)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs](https://mlsanity.com/paper/2307.07312/) (2023)
- [Learning to Transfer Prompts for Text Generation](https://mlsanity.com/paper/2205.01543/) (2022)
- [Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining](https://mlsanity.com/paper/2403.04780/) (2024)
- [Evaluating Generative Models for Graph-to-Text Generation](https://mlsanity.com/paper/2307.14712/) (2023)
- [Research Trends for the Interplay between Large Language Models and Knowledge Graphs](https://mlsanity.com/paper/2406.08223/) (2024)
- [ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models](https://mlsanity.com/paper/2210.04325/) (2022)
- [Efficient Knowledge Infusion via KG-LLM Alignment](https://mlsanity.com/paper/2406.03746/) (2024)
- [Graph Language Models](https://mlsanity.com/paper/2401.07105/) (2024)
- [Biomedical Knowledge Graph: A Survey of Domains, Tasks, and Real-World Applications](https://mlsanity.com/paper/2501.11632/) (2025)
- [Think While You Write: Hypothesis Verification Promotes Faithful Knowledge-to-Text Generation](https://mlsanity.com/paper/2311.09467/) (2023)
- [ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models](https://mlsanity.com/paper/2205.01523/) (2022)
- [Narrating Causal Graphs with Large Language Models](https://mlsanity.com/paper/2403.07118/) (2024)
- [Document Automation Architectures and Technologies: A Survey](https://mlsanity.com/paper/2109.11603/) (2021)
- [Improving Few-Shot Inductive Learning on Temporal Knowledge Graphs using Confidence-Augmented Reinforcement Learning](https://mlsanity.com/paper/2304.00613/) (2023)
- [AUGUST: an Automatic Generation Understudy for Synthesizing Conversational Recommendation Datasets](https://mlsanity.com/paper/2306.09631/) (2023)
- [Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning](https://mlsanity.com/paper/2308.05317/) (2023)
- [Biomedical Entity Linking with Triple-aware Pre-Training](https://mlsanity.com/paper/2308.14429/) (2023)
- [GraSAME: Injecting Token-Level Structural Information to Pretrained Language Models via Graph-guided Self-Attention Mechanism](https://mlsanity.com/paper/2404.06911/) (2024)
- [Syntax Controlled Knowledge Graph-to-Text Generation with Order and Semantic Consistency](https://mlsanity.com/paper/2207.00719/) (2022)
- [Document Automation Architectures: Updated Survey in Light of Large Language Models](https://mlsanity.com/paper/2308.09341/) (2023)
- [GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network](https://mlsanity.com/paper/2402.11709/) (2024)
- [Unsupervised Learning of Graph from Recipes](https://mlsanity.com/paper/2401.12088/) (2024)
- [Evaluating and Improving Graph to Text Generation with Large Language Models](https://mlsanity.com/paper/2501.14497/) (2025)
- [Deep Representation Learning for Electronic Design Automation](https://mlsanity.com/paper/2505.02105/) (2025)
- [Competency Questions as Executable Plans: a Controlled RAG Architecture for Cultural Heritage Storytelling](https://mlsanity.com/paper/2604.02545/) (2026)
- [Knowledge Graphs Meet Graph Neural Networks: A Comprehensive Survey](https://mlsanity.com/paper/2607.09666/) (2026)
- [GLM-RAG: Graph Language Models for Graph-Based Retrieval-Augmented Generation](https://mlsanity.com/paper/2607.28397/) (2026)

## Abstract (arXiv)

This paper studies how to automatically generate a natural language text that describes the facts in knowledge graph (KG). Considering the few-shot setting, we leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation. We make three major technical contributions, namely representation alignment for bridging the semantic gap between KG encodings and PLMs, relation-biased KG linearization for deriving better input representations, and multi-task learning for learning the correspondence between KG and text. Extensive experiments on three benchmark datasets have demonstrated the effectiveness of our model on KG-to-text generation task. In particular, our model outperforms all comparison methods on both fully-supervised and few-shot settings. Our code and datasets are available at https://github.com/RUCAIBox/Few-Shot-KG2Text.