mlsanityData to 7 Oct 2026

Evaluating Generative Models for Graph-to-Text Generation

arXiv 2307.14712 ↗First posted 27 Jul 2023cs.CL cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have been widely employed for graph-to-text generation tasks. However, the process of finetuning LLMs requires significant training resources and annotation work. In this paper, we explore the capability of generative models to generate descriptive text from graph data in a zero-shot setting. Specifically, we evaluate GPT-3 and ChatGPT on two graph-to-text datasets and compare their performance with that of finetuned LLM models such as T5 and BART. Our results demonstrate that generative models are capable of generating fluent and coherent text, achieving BLEU scores of 10.57 and 11.08 for the AGENDA and WebNLG datasets, respectively. However, our error analysis reveals that generative models still struggle with understanding the semantic relations between entities, and they also tend to generate text with hallucinations or irrelevant information. As a part of error analysis, we utilize BERT to detect machine-generated text and achieve high macro-F1 scores. We have made the text generated by generative models publicly available.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

24 of 37 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  7. 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 321
  8. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  9. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  10. 2019Text Generation from Knowledge Graphs with Graph Transformerscited by 122
  11. 2020Investigating Pretrained Language Models for Graph-to-Text Generationcited by 117
  12. 2023Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extractioncited by 89
12 more
  1. 2018Graph-to-Sequence Learning using Gated Graph Neural Networkscited by 61
  2. 2017Neural AMR: Sequence-to-Sequence Models for Parsing and Generationcited by 56
  3. 2018A Graph-to-Sequence Model for AMR-to-Text Generationcited by 56
  4. 2021Structural Adapters in Pretrained Language Models for AMR-to-text Generationcited by 48
  5. 2020CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Trainingcited by 38
  6. 2020Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelitycited by 35
  7. 2020Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphscited by 30
  8. 2021Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Modelscited by 29
  9. 2018Deep Graph Convolutional Encoders for Structured Data to Text Generationcited by 27
  10. 2019Enhancing AMR-to-Text Generation with Dual Graph Representationscited by 24
  11. 2019Structural Neural Encoders for AMR-to-text Generationcited by 19
  12. 2022Neural Pipeline for Zero-Shot Data-to-Text Generationcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 7; 2025: 3; 2026: 2.

16 papers.

Sort:

By area:

  1. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  2. 2023GPTEval: A Survey on Assessments of ChatGPT and GPT-4cited by 73
  3. 2024Can GNN be Good Adapter for LLMs?cited by 71
  4. 2024Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generationcited by 16
  5. 2024Can LLMs Understand Computer Networks? Towards a Virtual System Administratorcited by 11
  6. 2023Insights into Classifying and Mitigating LLMs' Hallucinationscited by 6
  7. 2024The Pitfalls of Defining Hallucinationcited by 5
  8. 2024Graph-Guided Textual Explanation Generation Frameworkcited by 3
  9. 2025Beyond Over-Refusal: Scenario-Based Diagnostics and Post-Hoc Mitigation for Exaggerated Refusals in LLMscited by 3
  10. 2024GraSAME: Injecting Token-Level Structural Information to Pretrained Language Models via Graph-guided Self-Attention Mechanismcited by 2
  11. 2025Can Hallucinations Help? Boosting LLMs for Drug Discoverycited by 2
  12. 2023Compositional Generalization for Data-to-Text Generationcited by 1
4 more
  1. 2024TrICy: Trigger-guided Data-to-text Generation with Intent aware Attention-Copycited by 0
  2. 2025Evaluating and Improving Graph to Text Generation with Large Language Modelscited by 0
  3. 2026What Gets Unmasked First? Trajectory Analysis of Diffusion Models for Graph-to-Text Generationcited by 0
  4. 2026Interpretable Unsupervised Community Detection with LLM-Symbolized Structured Processescited by 0