mlsanityData to 7 Oct 2026

STEB: Style Text Embedding Benchmark

arXiv 2606.31741 ↗First posted 30 Jun 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets. To bridge this gap, we introduce the Style Text Embedding Benchmark, a comprehensive open-source benchmark intended to standardize the evaluation of style embeddings. STEB encompasses 96 datasets across 7 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection, probing of linguistic features, and others. We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated. We open-source the STEB code base at: https://github.com/rrivera1849/STEB.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 94 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

40 of 94 references.

References · 40

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  5. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  6. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  7. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  8. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  9. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  10. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  11. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  12. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
28 more
  1. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  2. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 181
  3. 2023MAGE: Machine-generated Text Detection in the Wildcited by 130
  4. 2013A Computational Approach to Politeness with Application to Social Factorscited by 104
  5. 2020ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformationscited by 101
  6. 2022Same Author or Just Same Topic? Towards Content-Independent Style Representationscited by 83
  7. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  8. 2020Towards Ecologically Valid Research on Language User Interfacescited by 42
  9. 2024Few-Shot Detection of Machine-Generated Text using Style Representationscited by 40
  10. 2024Can Large Language Models Identify Authorship?cited by 40
  11. 2021StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfercited by 34
  12. 2024StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examplescited by 32
  13. 2023Learning Interpretable Style Embeddings via Prompting LLMscited by 29
  14. 2021A Deep Metric Learning Approach to Account Linkingcited by 24
  15. 2017Evaluating prose style transfer with the Biblecited by 23
  16. 2019Learning Invariant Representations of Social Media Userscited by 21
  17. 2023ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfercited by 18
  18. 2021Does It Capture STEL? A Modular, Similarity-based Linguistic Style Evaluation Frameworkcited by 17
  19. 2024TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddingscited by 16
  20. 2023Understanding writing style in social media with a supervised contrastively pre-trained transformercited by 11
  21. 2019(Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Annotated Stylistic Language Dataset with Multiple Personascited by 10
  22. 2023Learning to Generate Text in Arbitrary Writing Stylescited by 9
  23. 2024Separating Style from Substance: Enhancing Cross-Genre Authorship Attribution through Data Selection and Presentationcited by 7
  24. 2025Leveraging Multilingual Training for Authorship Representation: Enhancing Generalization across Languages and Domainscited by 7
  25. 2025Limits of LLM Text Detectors in Educationcited by 5
  26. 2026Overview of PAN 2026: Voight-Kampff Generative AI Detection, Text Watermarking, Multi-Author Writing Style Analysis, Generative Plagiarism Detection, and Reasoning Trajectory Detectioncited by 3
  27. 2025A stylometric analysis of speaker attribution from speech transcriptscited by 2
  28. 2025Cross-Genre Authorship Attribution via LLM-Based Retrieve-and-Rerankcited by 1

Built on by

Not cited yet.