On the interaction of automatic evaluation and task framing in headline style transfer
Not on the map. Why?
Abstract (arXiv)
An ongoing debate in the NLG community concerns the best way to evaluate systems, with human evaluation often being considered the most reliable method, compared to corpus-based metrics. However, tasks involving subtle textual differences, such as style transfer, tend to be hard for humans to perform. In this paper, we propose an evaluation method for this task based on purposely-trained classifiers, showing that it better reflects system differences than traditional metrics such as BLEU and ROUGE.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 10 of its 23 references that are in the map. First posted on arXiv 5 Jan 2021.
Builds on
10 of 23 references.
References · 10
- 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
- 2020Evaluation of Text Generation: A Surveycited by 277
- 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
- 2017Why We Need New Evaluation Metrics for NLGcited by 233
- 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 211
- 2017Style Transfer in Text: Exploration and Evaluationcited by 146
- 2018RankME: Reliable Human Ratings for Natural Language Generationcited by 69
- 2019A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfercited by 65
- 2019Evaluating Style Transfer for Textcited by 61
Built on by
Citing papers per year: 2022: 1.
1 paper.