mlsanityData to 7 Oct 2026

Does It Capture STEL? A Modular, Similarity-based Linguistic Style Evaluation Framework

arXiv 2109.04817 ↗First posted 10 Sep 2021cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Style is an integral part of natural language. However, evaluation methods for style measures are rare, often task-specific and usually do not control for content. We propose the modular, fine-grained and content-controlled similarity-based STyle EvaLuation framework (STEL) to test the performance of any model that can compare two sentences on style. We illustrate STEL with two general dimensions of style (formal/informal and simple/complex) as well as two specific characteristics of style (contrac'tion and numb3r substitution). We find that BERT-based methods outperform simple versions of commonly used style measures like 3-grams, punctuation frequency and LIWC-based approaches. We invite the addition of further tasks and task instances to STEL and hope to facilitate the improvement of style-sensitive measures.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 26 of its 78 references that are in the map. First posted on arXiv 10 Sep 2021.

Builds on

26 of 78 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  7. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  8. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  9. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 291
  10. 2020The Pushshift Reddit Datasetcited by 289
  11. 2016A Persona-Based Neural Conversation Modelcited by 221
  12. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
14 more
  1. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 181
  2. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 147
  3. 2013A Computational Approach to Politeness with Application to Social Factorscited by 104
  4. 2020Politeness Transfer: A Tag and Generate Approachcited by 78
  5. 2016Personalized Machine Translation: Preserving Original Author Traitscited by 47
  6. 2019Explainable Authorship Verification in Social Media via Attention-based Similarity Learningcited by 36
  7. 2015Computational Sociolinguistics: A Surveycited by 35
  8. 2020The Importance of Suppressing Domain Style in Authorship Analysiscited by 18
  9. 2016Learning Stylometric Representations for Authorship Analysiscited by 14
  10. 2019metric-learn: Metric Learning Algorithms in Pythoncited by 11
  11. 2019(Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Annotated Stylistic Language Dataset with Multiple Personascited by 10
  12. 2020Deep Bayes Factor Scoring for Authorship Verificationcited by 7
  13. 2019Low-Level Linguistic Controls for Style Transfer and Content Preservationcited by 3
  14. 2018Unsupervised Learning of Style-sensitive Word Vectorscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 3; 2025: 2; 2026: 4.

17 papers.

Sort:

By area:

  1. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
  2. 2022Same Author or Just Same Topic? Towards Content-Independent Style Representationscited by 83
  3. 2024StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examplescited by 32
  4. 2023Learning Interpretable Style Embeddings via Prompting LLMscited by 29
  5. 2023A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Textscited by 23
  6. 2023Can Authorship Representation Learning Capture Stylistic Features?cited by 17
  7. 2023ICLEF: In-Context Learning with Expert Feedback for Explainable Style Transfercited by 5
  8. 2025Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processingcited by 5
  9. 2024WordDecipher: Enhancing Digital Workspace Communication with Explainable AI for Non-native English Speakerscited by 2
  10. 2023SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative Referencescited by 1
  11. 2023Authorship identification under domain shift: a survey of stylistic measures and learned author representationscited by 1
  12. 2024HALvest-Contrastive: Retrieval-Like Authorship Attribution with Patch-Level Late Interactioncited by 1
5 more
  1. 2026StoryLens: Preference-Aligned Story Rewriting via Context-Aware Narrative Enrichmentcited by 1
  2. 2025iBERT: Interpretable Embeddings via Sense Decompositioncited by 0
  3. 2026Where Does Authorship Signal Emerge in Encoder-Based Language Models?cited by 0
  4. 2026Unsupervised Style Representation Learning for AI-Text Detection via Paraphrase Inversioncited by 0
  5. 2026STEB: Style Text Embedding Benchmarkcited by 0