# GPT-too: A language-model-first approach for AMR-to-text generation

arXiv: https://arxiv.org/abs/2005.09123 · HTML: https://mlsanity.com/paper/2005.09123/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 30 of 38)

- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2105.10311/) (2021)
- [Tailor: Generating and Perturbing Text with Semantic Controls](https://mlsanity.com/paper/2107.07150/) (2021)
- [JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs](https://mlsanity.com/paper/2106.10502/) (2021)
- [Graph Pre-training for AMR Parsing and Generation](https://mlsanity.com/paper/2203.07836/) (2022)
- [Structural Adapters in Pretrained Language Models for AMR-to-text Generation](https://mlsanity.com/paper/2103.09120/) (2021)
- [Coherence boosting: When your pretrained language model is not paying enough attention](https://mlsanity.com/paper/2110.08294/) (2021)
- [Maximum Bayes Smatch Ensemble Distillation for AMR Parsing](https://mlsanity.com/paper/2112.07790/) (2021)
- [DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations](https://mlsanity.com/paper/2203.09711/) (2022)
- [Structural Information Preserving for Graph-to-Text Generation](https://mlsanity.com/paper/2102.06749/) (2021)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generation](https://mlsanity.com/paper/2204.06674/) (2022)
- [Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling](https://mlsanity.com/paper/2102.03551/) (2021)
- [Constrained Text Generation with Global Guidance -- Case Study on CommonGen](https://mlsanity.com/paper/2103.07170/) (2021)
- [Graphine: A Dataset for Graph-aware Terminology Definition Generation](https://mlsanity.com/paper/2109.04018/) (2021)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Bootstrapping Multilingual AMR with Contextual Word Alignments](https://mlsanity.com/paper/2102.02189/) (2021)
- [What Makes Data-to-Text Generation Hard for Pretrained Language Models?](https://mlsanity.com/paper/2205.11505/) (2022)
- [Self-supervised Graph Masking Pre-training for Graph-to-Text Generation](https://mlsanity.com/paper/2210.10599/) (2022)
- [Contextualized Scene Imagination for Generative Commonsense Reasoning](https://mlsanity.com/paper/2112.06318/) (2021)
- [LOGEN: Few-shot Logical Knowledge-Conditioned Text Generation with Self-training](https://mlsanity.com/paper/2112.01404/) (2021)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)
- [A Systematic Review of Data-to-Text NLG](https://mlsanity.com/paper/2402.08496/) (2024)
- [Narrating Causal Graphs with Large Language Models](https://mlsanity.com/paper/2403.07118/) (2024)
- [Smelting Gold and Silver for Improved Multilingual AMR-to-Text Generation](https://mlsanity.com/paper/2109.03808/) (2021)
- [Document Automation Architectures and Technologies: A Survey](https://mlsanity.com/paper/2109.11603/) (2021)
- [On the Role of Pre-trained Language Models in Word Ordering: A Case Study with BART](https://mlsanity.com/paper/2204.07367/) (2022)
- [GraSAME: Injecting Token-Level Structural Information to Pretrained Language Models via Graph-guided Self-Attention Mechanism](https://mlsanity.com/paper/2404.06911/) (2024)
- [An evaluation of LLMs and Google Translate for translation of selected Indian languages via sentiment and semantic analyses](https://mlsanity.com/paper/2503.21393/) (2025)
- [Survey of Abstract Meaning Representation: Then, Now, Future](https://mlsanity.com/paper/2505.03229/) (2025)

## Abstract (arXiv)

Meaning Representations (AMRs) are broad-coverage sentence-level semantic graphs. Existing approaches to generating text from AMR have focused on training sequence-to-sequence or graph-to-sequence models on AMR annotated data only. In this paper, we propose an alternative approach that combines a strong pre-trained language model with cycle consistency-based re-scoring. Despite the simplicity of the approach, our experimental results show these models outperform all previous techniques on the English LDC2017T10dataset, including the recent use of transformer architectures. In addition to the standard evaluation metrics, we provide human evaluation experiments that further substantiate the strength of our approach.