# Style-Specific Neurons for Steering LLMs in Text Style Transfer

arXiv: https://arxiv.org/abs/2410.00593 · HTML: https://mlsanity.com/paper/2410.00593/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 1 Oct 2024.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [How do Large Language Models Handle Multilingualism?](https://mlsanity.com/paper/2402.18815/) (2024)
- [Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models](https://mlsanity.com/paper/2402.16438/) (2024)
- [On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons](https://mlsanity.com/paper/2404.02431/) (2024)
- [Politeness Transfer: A Tag and Generate Approach](https://mlsanity.com/paper/2004.14257/) (2020)
- [On the Relation between Sensitivity and Accuracy in In-context Learning](https://mlsanity.com/paper/2209.07661/) (2022)
- [What does the Knowledge Neuron Thesis Have to do with Knowledge?](https://mlsanity.com/paper/2405.02421/) (2024)
- [PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits](https://mlsanity.com/paper/2404.06453/) (2024)
- [Neuron Specialization: Leveraging intrinsic task modularity for multilingual machine translation](https://mlsanity.com/paper/2404.11201/) (2024)
- [Prompt-Based Editing for Text Style Transfer](https://mlsanity.com/paper/2301.11997/) (2023)
- [Exploring Methods for Cross-lingual Text Style Transfer: The Case of Text Detoxification](https://mlsanity.com/paper/2311.13937/) (2023)
- [Text Style Transfer Evaluation Using Large Language Models](https://mlsanity.com/paper/2308.13577/) (2023)
- [Distilling Text Style Transfer With Self-Explanation From LLMs](https://mlsanity.com/paper/2403.01106/) (2024)
- [Improving Both Domain Robustness and Domain Adaptability in Machine Translation](https://mlsanity.com/paper/2112.08288/) (2021)
- [Text Detoxification as Style Transfer in English and Hindi](https://mlsanity.com/paper/2402.07767/) (2024)
- [Multilingual Text Style Transfer: Datasets & Models for Indian Languages](https://mlsanity.com/paper/2405.20805/) (2024)
- [Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models](https://mlsanity.com/paper/2402.09099/) (2024)
- [Unsupervised Text Style Transfer via LLMs and Attention Masking with Multi-way Interactions](https://mlsanity.com/paper/2402.13647/) (2024)
- [Prefix-Tuning Based Unsupervised Text Style Transfer](https://mlsanity.com/paper/2310.14599/) (2023)
- [Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation](https://mlsanity.com/paper/2305.12786/) (2023)
- [LMStyle Benchmark: Evaluating Text Style Transfer for Chatbots](https://mlsanity.com/paper/2403.08943/) (2024)

## Built on by (most cited first, 24 of 24)

- [SAEs Are Good for Steering -- If You Select the Right Features](https://mlsanity.com/paper/2505.20063/) (2025)
- [SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers](https://mlsanity.com/paper/2504.00255/) (2025)
- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [Language-specific Neurons Do Not Facilitate Cross-Lingual Transfer](https://mlsanity.com/paper/2503.17456/) (2025)
- [SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models](https://mlsanity.com/paper/2510.26769/) (2025)
- [PERSA: Reinforcement Learning for Professor-Style Personalized Feedback with LLMs](https://mlsanity.com/paper/2605.01123/) (2026)
- [Activation-Space Personality Steering: Hybrid Layer Selection for Stable Trait Control in LLMs](https://mlsanity.com/paper/2511.03738/) (2025)
- [LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxification](https://mlsanity.com/paper/2506.01484/) (2025)
- [Finding Culture-Sensitive Neurons in Vision-Language Models](https://mlsanity.com/paper/2510.24942/) (2025)
- [Neuron Empirical Gradient: Discovering and Quantifying Neurons Global Linear Controllability](https://mlsanity.com/paper/2412.18053/) (2024)
- [Mind the Style Gap: Meta-Evaluation of Style and Attribute Transfer Metrics](https://mlsanity.com/paper/2502.15022/) (2025)
- [Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models](https://mlsanity.com/paper/2605.30717/) (2026)
- [Transferring Extreme Subword Style Using Ngram Model-Based Logit Scaling](https://mlsanity.com/paper/2503.08550/) (2025)
- [Implementing Long Text Style Transfer with LLMs through Dual-Layered Sentence and Paragraph Structure Extraction and Mapping](https://mlsanity.com/paper/2505.07888/) (2025)
- [SynDec: A Synthesize-then-Decode Approach for Arbitrary Textual Style Transfer via Large Language Models](https://mlsanity.com/paper/2505.12821/) (2025)
- [Analyzing Bias in False Refusal Behavior of Large Language Models for Hate Speech Detoxification](https://mlsanity.com/paper/2601.08668/) (2026)
- [StoryLens: Preference-Aligned Story Rewriting via Context-Aware Narrative Enrichment](https://mlsanity.com/paper/2605.28073/) (2026)
- [SurvSurf: a partially monotonic neural network for first-hitting time prediction of intermittently observed discrete and continuous sequential events](https://mlsanity.com/paper/2504.04997/) (2025)
- [A Universal Vibe? Finding and Controlling Language-Agnostic Informal Register with SAEs](https://mlsanity.com/paper/2603.26236/) (2026)
- [Casual as an Anchor: Resolving Supervision Misalignment in Formality Transfer Dataset](https://mlsanity.com/paper/2605.29365/) (2026)
- [Activation-Guided Neuron Intervention to Induce Alzheimer's-Related Computational Language Phenotypes in a Large Language Model](https://mlsanity.com/paper/2608.03067/) (2026)
- [Do Recipes Have Personas? Characterizing and Generating Creator Style in Attributed Procedural Graphs](https://mlsanity.com/paper/2608.24369/) (2026)
- [Attribute-Based Activation Steering of LLMs for Group-Specific Explanation Generation](https://mlsanity.com/paper/2608.29215/) (2026)
- [In-Place Instruction Following in Diffusion Language Models](https://mlsanity.com/paper/2609.07160/) (2026)

## Abstract (arXiv)

Text style transfer (TST) aims to modify the style of a text without altering its original meaning. Large language models (LLMs) demonstrate superior performance across multiple tasks, including TST. However, in zero-shot setups, they tend to directly copy a significant portion of the input text to the output without effectively changing its style. To enhance the stylistic variety and fluency of the text, we present sNeuron-TST, a novel approach for steering LLMs using style-specific neurons in TST. Specifically, we identify neurons associated with the source and target styles and deactivate source-style-only neurons to give target-style words a higher probability, aiming to enhance the stylistic diversity of the generated text. However, we find that this deactivation negatively impacts the fluency of the generated text, which we address by proposing an improved contrastive decoding method that accounts for rapid token probability shifts across layers caused by deactivated source-style neurons. Empirical experiments demonstrate the effectiveness of the proposed method on six benchmarks, encompassing formality, toxicity, politics, politeness, authorship, and sentiment.