mlsanityData to 7 Oct 2026

Challenging Assumptions in Learning Generic Text Style Embeddings

arXiv 2501.16073 ↗First posted 27 Jan 2025cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advancements in language representation learning primarily emphasize language modeling for deriving meaningful representations, often neglecting style-specific considerations. This study addresses this gap by creating generic, sentence-level style embeddings crucial for style-centric tasks. Our approach is grounded on the premise that low-level text style changes can compose any high-level style. We hypothesize that applying this concept to representation learning enables the development of versatile text style embeddings. By fine-tuning a general-purpose text encoder using contrastive learning and standard cross-entropy loss, we aim to capture these low-level style shifts, anticipating that they offer insights applicable to high-level text styles. The outcomes prompt us to reconsider the underlying assumptions as the results do not always show that the learned style representations capture high-level text styles.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 27 Jan 2025.

Builds on

19 of 26 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  5. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  6. 2017Toward Controlled Generation of Textcited by 271
  7. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  8. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
  9. 2018An efficient framework for learning sentence representationscited by 197
  10. 2017Style Transfer in Text: Exploration and Evaluationcited by 145
  11. 2018Disentangled Representation Learning for Non-Parallel Text Style Transfercited by 127
  12. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
7 more
  1. 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
  2. 2020Improving Disentangled Text Representation Learning with Information-Theoretic Guidancecited by 52
  3. 2024Few-Shot Detection of Machine-Generated Text using Style Representationscited by 40
  4. 2021StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfercited by 34
  5. 2024StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examplescited by 32
  6. 2017Evaluating prose style transfer with the Biblecited by 23
  7. 2023A Call for Standardization and Validation of Text Style Transfer Evaluationcited by 13

Built on by

Not cited yet.