mlsanityData to 7 Oct 2026

Investigating Pretrained Language Models for Graph-to-Text Generation

arXiv 2007.08426 ↗First posted 16 Jul 2020cs.CLcited by 117

Not on the map. Why?

Abstract (arXiv)

Graph-to-text generation aims to generate fluent texts from graph-based data. In this paper, we investigate two recently proposed pretrained language models (PLMs) and analyze the impact of different task-adaptive pretraining strategies for PLMs in graph-to-text generation. We present a study across three graph domains: meaning representations, Wikipedia knowledge graphs (KGs) and scientific KGs. We show that the PLMs BART and T5 achieve new state-of-the-art results and that task-adaptive pretraining strategies improve their performance even further. In particular, we report new state-of-the-art BLEU scores of 49.72 on LDC2017T10, 59.70 on WebNLG, and 25.66 on AGENDA datasets - a relative improvement of 31.8%, 4.5%, and 42.4%, respectively. In an extensive analysis, we identify possible reasons for the PLMs' success on graph-to-text tasks. We find evidence that their knowledge about true facts helps them perform well even when the input graph representation is reduced to a simple bag of node and edge labels.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 33; 2023: 33; 2024: 20; 2025: 3; 2026: 6.

117 papers.

Sort:

By area:

  1. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  2. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  3. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  4. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  5. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  6. 2021Pretrained Language Models for Text Generation: A Surveycited by 95
  7. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  8. 2023Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answeringcited by 64
  9. 2021JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphscited by 58
  10. 2022Graph Pre-training for AMR Parsing and Generationcited by 55
  11. 2021Plan-then-Generate: Controlled Data-to-Text Generation via Planningcited by 52
  12. 2021Structural Adapters in Pretrained Language Models for AMR-to-text Generationcited by 48
38 more
  1. 2021ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Miningcited by 37
  2. 2021Open Domain Question Answering with A Unified Knowledge Interfacecited by 32
  3. 2021Relational World Knowledge Representation in Contextual Language Models: A Reviewcited by 31
  4. 2021Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Modelscited by 29
  5. 2021Maximum Bayes Smatch Ensemble Distillation for AMR Parsingcited by 27
  6. 2022DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulationscited by 25
  7. 2023Learning Probabilistic Symmetrization for Architecture Agnostic Equivariancecited by 25
  8. 2021Logic-Consistency Text Generation from Semantic Parsescited by 23
  9. 2021Control Prefixes for Parameter-Efficient Text Generationcited by 23
  10. 2022CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generationcited by 23
  11. 2023Fairness-Aware Graph Neural Networks: A Surveycited by 22
  12. 2021Few-Shot Table-to-Text Generation with Prototype Memorycited by 21
  13. 2024Graph-Based Retriever Captures the Long Tail of Biomedical Knowledgecited by 20
  14. 2024Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Miningcited by 20
  15. 2022Neural Pipeline for Zero-Shot Data-to-Text Generationcited by 19
  16. 2021ReGen: Reinforcement Learning for Text and Knowledge Base Generation using Pretrained Language Modelscited by 16
  17. 2022GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generationcited by 16
  18. 2023Evaluating Generative Models for Graph-to-Text Generationcited by 16
  19. 2024Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generationcited by 16
  20. 2022Knowledge Graph Generation From Textcited by 15
  21. 2021EventNarrative: A large-scale Event-centric Dataset for Knowledge Graph-to-Text Generationcited by 14
  22. 2022ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Modelscited by 14
  23. 2024Research Trends for the Interplay between Large Language Models and Knowledge Graphscited by 14
  24. 2021Learning to Reason for Text Generation from Scientific Tablescited by 13
  25. 2023Sequential Query Encoding For Complex Query Answering on Knowledge Graphscited by 13
  26. 2024Unifying Structured Data as Graph for Data-to-Text Pre-Trainingcited by 13
  27. 2022QASem Parsing: Text-to-text Modeling of QA-based Semanticscited by 12
  28. 2022Interpretable AMR-Based Question Decomposition for Multi-hop Question Answeringcited by 12
  29. 2021Stage-wise Fine-tuning for Graph-to-Text Generationcited by 11
  30. 2021DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generationcited by 11
  31. 2021Element Intervention for Open Relation Extractioncited by 11
  32. 2022Context-Tuning: Learning Contextualized Prompts for Natural Language Generationcited by 11
  33. 2023ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translationcited by 11
  34. 2021Improving Scene Graph Classification by Exploiting Knowledge from Textscited by 10
  35. 2022Innovations in Neural Data-to-text Generation: A Surveycited by 10
  36. 2023AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generationcited by 10
  37. 2024Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answeringcited by 9
  38. 2022What Makes Data-to-Text Generation Hard for Pretrained Language Models?cited by 8

50 most cited of 117 All 117 (JSON)