# Learning Interpretable Style Embeddings via Prompting LLMs

arXiv: https://arxiv.org/abs/2305.12696 · HTML: https://mlsanity.com/paper/2305.12696/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 74 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 30 of 45)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013)
- [Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor](https://mlsanity.com/paper/2212.09689/) (2022)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Style Transfer in Text: Exploration and Evaluation](https://mlsanity.com/paper/1711.06861/) (2017)
- [Disentangled Representation Learning for Non-Parallel Text Style Transfer](https://mlsanity.com/paper/1808.04339/) (2018)
- [Style Transfer Through Back-Translation](https://mlsanity.com/paper/1804.09000/) (2018)
- [Automatically Neutralizing Subjective Bias in Text](https://mlsanity.com/paper/1911.09709/) (2019)
- [ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations](https://mlsanity.com/paper/2005.00481/) (2020)

## Built on by (most cited first, 30 of 30)

- [Can Large Language Models Identify Authorship?](https://mlsanity.com/paper/2403.08213/) (2024)
- [StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples](https://mlsanity.com/paper/2410.12757/) (2024)
- [CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following](https://mlsanity.com/paper/2403.03129/) (2024)
- [Bayesian Concept Bottleneck Models with LLM Priors](https://mlsanity.com/paper/2410.15555/) (2024)
- [Human-AI Co-design for Clinical Prediction Models](https://mlsanity.com/paper/2601.09072/) (2026)
- [Latent Space Interpretation for Stylistic Analysis and Explainable Authorship Attribution](https://mlsanity.com/paper/2409.07072/) (2024)
- [HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation](https://mlsanity.com/paper/2505.00038/) (2025)
- [Fundamental Limits of Perfect Concept Erasure](https://mlsanity.com/paper/2503.20098/) (2025)
- [ICLEF: In-Context Learning with Expert Feedback for Explainable Style Transfer](https://mlsanity.com/paper/2309.08583/) (2023)
- [Mitigating Sycophancy in Decoder-Only Transformer Architectures: Synthetic Data Intervention](https://mlsanity.com/paper/2411.10156/) (2024)
- [PRISM: A Framework for Producing Interpretable Political Bias Embeddings with Political-Aware Cross-Encoder](https://mlsanity.com/paper/2505.24646/) (2025)
- [Speaker Verification in Agent-Generated Conversations](https://mlsanity.com/paper/2405.10150/) (2024)
- [Residualized Similarity for Faithfully Explainable Authorship Verification](https://mlsanity.com/paper/2510.05362/) (2025)
- [CAVE: Controllable Authorship Verification Explanations](https://mlsanity.com/paper/2406.16672/) (2024)
- [Personalized Benchmarking: Evaluating LLMs by Individual Preferences](https://mlsanity.com/paper/2604.18943/) (2026)
- [WordDecipher: Enhancing Digital Workspace Communication with Explainable AI for Non-native English Speakers](https://mlsanity.com/paper/2404.07005/) (2024)
- [The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributes](https://mlsanity.com/paper/2504.17300/) (2025)
- [IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variation](https://mlsanity.com/paper/2604.04704/) (2026)
- [Authorship identification under domain shift: a survey of stylistic measures and learned author representations](https://mlsanity.com/paper/2310.00436/) (2023)
- [mStyleDistance: Multilingual Style Embeddings and their Evaluation](https://mlsanity.com/paper/2502.15168/) (2025)
- [LDIR: Low-Dimensional Dense and Interpretable Text Embeddings with Relative Representations](https://mlsanity.com/paper/2505.10354/) (2025)
- [What's in a prompt? Language models encode literary style in prompt embeddings](https://mlsanity.com/paper/2505.17071/) (2025)
- [Better Call Claude: Can LLMs Detect Changes of Writing Style?](https://mlsanity.com/paper/2508.00680/) (2025)
- [Stylometry recognizes human and LLM-generated texts in short samples](https://mlsanity.com/paper/2507.00838/) (2025)
- [XAM: Interactive Explainability for Authorship Attribution Models](https://mlsanity.com/paper/2512.06924/) (2025)
- [Differentially-Private Text Rewriting reshapes Linguistic Style](https://mlsanity.com/paper/2604.26656/) (2026)
- [Interpreting Style Representations via Style-Eliciting Prompts](https://mlsanity.com/paper/2606.05716/) (2026)
- [STEB: Style Text Embedding Benchmark](https://mlsanity.com/paper/2606.31741/) (2026)
- [Attribute-Based Activation Steering of LLMs for Group-Specific Explanation Generation](https://mlsanity.com/paper/2608.29215/) (2026)
- [Author Representation Strategies for Zero-Shot Authorship Attribution: A Comparative Study of LLM-Based and Embedding-Based Approaches](https://mlsanity.com/paper/2610.03531/) (2026)

## Abstract (arXiv)

Style representation learning builds content-independent representations of author style in text. Stylometry, the analysis of style in text, is often performed by expert forensic linguists and no large dataset of stylometric annotations exists for training. Current style representation learning uses neural methods to disentangle style from content to create style vectors, however, these approaches result in uninterpretable representations, complicating their usage in downstream applications like authorship attribution where auditing and explainability is critical. In this work, we use prompting to perform stylometry on a large number of texts to create a synthetic dataset and train human-interpretable style representations we call LISA embeddings. We release our synthetic stylometry dataset and our interpretable style models as resources.