# The Daunting Task of Real-World Textual Style Transfer Auto-Evaluation

arXiv: https://arxiv.org/abs/1910.03747 · HTML: https://mlsanity.com/paper/1910.03747/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [Attacking Misinformation Detection Using Adversarial Examples Generated by Language Models](https://mlsanity.com/paper/2410.20940/) (2024)
- [StyleKQC: A Style-Variant Paraphrase Corpus for Korean Questions and Commands](https://mlsanity.com/paper/2103.13439/) (2021)
- [Investigating Text Simplification Evaluation](https://mlsanity.com/paper/2107.13662/) (2021)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)
- [Text Style Transfer: An Introductory Overview](https://mlsanity.com/paper/2407.14822/) (2024)
- [Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMs](https://mlsanity.com/paper/2601.18483/) (2026)
- [RT-SFT: Text Style Transfer from Non-Parallel Corpora by Roundtrip Translation](https://mlsanity.com/paper/2602.15013/) (2026)

## Abstract (arXiv)

The difficulty of textual style transfer lies in the lack of parallel corpora. Numerous advances have been proposed for the unsupervised generation. However, significant problems remain with the auto-evaluation of style transfer tasks. Based on the summary of Pang and Gimpel (2018) and Mir et al. (2019), style transfer evaluations rely on three criteria: style accuracy of transferred sentences, content similarity between original and transferred sentences, and fluency of transferred sentences. We elucidate the problematic current state of style transfer research. Given that current tasks do not represent real use cases of style transfer, current auto-evaluation approach is flawed. This discussion aims to bring researchers to think about the future of style transfer and style transfer evaluation research.