# TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings

arXiv: https://arxiv.org/abs/2406.15586 · HTML: https://mlsanity.com/paper/2406.15586/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 21 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment](https://mlsanity.com/paper/2304.06767/) (2023)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark](https://mlsanity.com/paper/2310.18018/) (2023)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Are Emergent Abilities in Large Language Models just In-Context Learning?](https://mlsanity.com/paper/2309.01809/) (2023)
- [Same Author or Just Same Topic? Towards Content-Independent Style Representations](https://mlsanity.com/paper/2204.04907/) (2022)
- [Controlled Text Generation as Continuous Optimization with Multiple Constraints](https://mlsanity.com/paper/2108.01850/) (2021)
- [Mix and Match: Learning-free Controllable Text Generation using Energy Language Models](https://mlsanity.com/paper/2203.13299/) (2022)
- [Text Detoxification using Large Pre-trained Neural Models](https://mlsanity.com/paper/2109.08914/) (2021)
- [Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models](https://mlsanity.com/paper/2205.11503/) (2022)
- [TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling](https://mlsanity.com/paper/2010.03802/) (2020)
- [A Deep Metric Learning Approach to Account Linking](https://mlsanity.com/paper/2105.07263/) (2021)
- [ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer](https://mlsanity.com/paper/2308.15459/) (2023)
- [Distilling Text Style Transfer With Self-Explanation From LLMs](https://mlsanity.com/paper/2403.01106/) (2024)
- [Detecting Text Formality: A Study of Text Classification Approaches](https://mlsanity.com/paper/2204.08975/) (2022)
- [Learning to Generate Text in Arbitrary Writing Styles](https://mlsanity.com/paper/2312.17242/) (2023)
- [T-STAR: Truthful Style Transfer using AMR Graph as Intermediate Representation](https://mlsanity.com/paper/2212.01667/) (2022)

## Built on by (most cited first, 17 of 17)

- [Leveraging Multilingual Training for Authorship Representation: Enhancing Generalization across Languages and Domains](https://mlsanity.com/paper/2509.16531/) (2025)
- [Steering Large Language Models with Register Analysis for Arbitrary Style Transfer](https://mlsanity.com/paper/2505.00679/) (2025)
- [SynthDetoxM: Modern LLMs are Few-Shot Parallel Detoxification Data Annotators](https://mlsanity.com/paper/2502.06394/) (2025)
- [Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning](https://mlsanity.com/paper/2502.08972/) (2025)
- [Evaluating Style-Personalized Text Generation: Challenges and Directions](https://mlsanity.com/paper/2508.06374/) (2025)
- [Reranking-based Generation for Unbiased Perspective Summarization](https://mlsanity.com/paper/2506.15925/) (2025)
- [AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing](https://mlsanity.com/paper/2603.23069/) (2026)
- [SAG: Style-Aligned Article Generation via Model Collaboration](https://mlsanity.com/paper/2410.03137/) (2024)
- [StyleAdaptedLM: Enhancing Instruction Following Models with Efficient Stylistic Transfer](https://mlsanity.com/paper/2507.18294/) (2025)
- [RT-SFT: Text Style Transfer from Non-Parallel Corpora by Roundtrip Translation](https://mlsanity.com/paper/2602.15013/) (2026)
- [Interpreting Style Representations via Style-Eliciting Prompts](https://mlsanity.com/paper/2606.05716/) (2026)
- [STEB: Style Text Embedding Benchmark](https://mlsanity.com/paper/2606.31741/) (2026)
- [Small Is Enough: Per-User Style Rewriting of AI-Edited Text via LoRA Adapters](https://mlsanity.com/paper/2607.29238/) (2026)
- [HyperStyler: Low-resource Authorship Style Transfer via Context-aware Style Navigation and Hypernetworks](https://mlsanity.com/paper/2609.02772/) (2026)
- [Can Activation Steering Capture Multidimensional Authorship Style?](https://mlsanity.com/paper/2609.04792/) (2026)
- [Forging LLM Authorship Fingerprints with Targeted Rewriting](https://mlsanity.com/paper/2609.38831/) (2026)
- [Predicting Steering Vectors and Adapter Weights for Few-Shot Author-Style Transfer](https://mlsanity.com/paper/2610.03163/) (2026)

## Abstract (arXiv)

The goal of text style transfer is to transform the style of texts while preserving their original meaning, often with only a few examples of the target style. Existing style transfer methods generally rely on the few-shot capabilities of large language models or on complex controllable text generation approaches that are inefficient and underperform on fluency metrics. We introduce TinyStyler, a lightweight but effective approach, which leverages a small language model (800M params) and pre-trained authorship embeddings to perform efficient, few-shot text style transfer. We evaluate on the challenging task of authorship style transfer and find TinyStyler outperforms strong approaches such as GPT-4. We also evaluate TinyStyler's ability to perform text attribute style transfer (formal leftrightarrow informal) with automatic and human evaluations and find that the approach outperforms recent controllable text generation methods. Our model has been made publicly available at https://huggingface.co/tinystyler/tinystyler .