mlsanityData to 7 Oct 2026

A Human Evaluation of AMR-to-English Generation Systems

arXiv 2004.06814 ↗First posted 14 Apr 2020cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Most current state-of-the art systems for generating English text from Abstract Meaning Representation (AMR) have been evaluated only using automated metrics, such as BLEU, which are known to be problematic for natural language generation. In this work, we present the results of a new human evaluation which collects fluency and adequacy scores, as well as categorization of error types, for several recent AMR generation systems. We discuss the relative quality of these systems and how our results compare to those of automatic metrics, finding that while the metrics are mostly successful in ranking systems overall, collecting human judgments allows for more nuanced comparisons. We also analyze common errors made by these systems.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 4; 2023: 0; 2024: 1; 2025: 3; 2026: 2.

13 papers.

Sort:

  1. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  2. 2021ToxCCIn: Toxic Content Classification with Interpretabilitycited by 10
  3. 2022Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarizationcited by 10
  4. 2021Biomedical Data-to-Text Generation via Fine-Tuning Transformerscited by 9
  5. 2024Natural Language Processing RELIES on Linguisticscited by 9
  6. 2022A Dynamic, Interpreted CheckList for Meaning-oriented NLG Metric Evaluation -- through the Lens of Semantic Similarity Ratingcited by 5
  7. 2025Sentence Smith: Controllable Edits for Evaluating Text Embeddingscited by 4
  8. 2022Spanish Abstract Meaning Representation: Annotation of a General Corpuscited by 2
  9. 2025Generating Text from Uniform Meaning Representationcited by 1
  10. 2022A Survey : Neural Networks for AMR-to-Textcited by 0
  11. 2025Abstract Meaning Representation for Hospital Discharge Summarizationcited by 0
  12. 2026Graph Alignment Topology as an Inductive Bias for Grounding Detectioncited by 0
1 more
  1. 2026Toward Complete Hospital Discharge Summarization with Abstract Meaning Representationcited by 0