# SETTP: Style Extraction and Tunable Inference via Dual-level Transferable Prompt Learning

arXiv: https://arxiv.org/abs/2407.15556 · HTML: https://mlsanity.com/paper/2407.15556/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 31 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Key-Value Memory Networks for Directly Reading Documents](https://mlsanity.com/paper/1606.03126/) (2016)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts](https://mlsanity.com/paper/2205.11961/) (2022)
- [Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study](https://mlsanity.com/paper/2304.00723/) (2023)
- [Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfer](https://mlsanity.com/paper/1908.09368/) (2019)
- [Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach](https://mlsanity.com/paper/1805.05181/) (2018)
- [Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer](https://mlsanity.com/paper/1805.07685/) (2018)
- [TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling](https://mlsanity.com/paper/2010.03802/) (2020)
- [Learning to Transfer Prompts for Text Generation](https://mlsanity.com/paper/2205.01543/) (2022)
- [A Call for Standardization and Validation of Text Style Transfer Evaluation](https://mlsanity.com/paper/2306.00539/) (2023)
- [Text Style Transfer Evaluation Using Large Language Models](https://mlsanity.com/paper/2308.13577/) (2023)
- [SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains](https://mlsanity.com/paper/2302.06868/) (2023)
- [Style is NOT a single variable: Case Studies for Cross-Style Language Understanding](https://mlsanity.com/paper/1911.03663/) (2019)
- [Specializing Small Language Models towards Complex Style Transfer via Latent Attribute Pre-Training](https://mlsanity.com/paper/2309.10929/) (2023)

## Abstract (arXiv)

Text style transfer, an important research direction in natural language processing, aims to adapt the text to various preferences but often faces challenges with limited resources. In this work, we introduce a novel method termed Style Extraction and Tunable Inference via Dual-level Transferable Prompt Learning (SETTP) for effective style transfer in low-resource scenarios. First, SETTP learns source style-level prompts containing fundamental style characteristics from high-resource style transfer. During training, the source style-level prompts are transferred through an attention module to derive a target style-level prompt for beneficial knowledge provision in low-resource style transfer. Additionally, we propose instance-level prompts obtained by clustering the target resources based on the semantic content to reduce semantic bias. We also propose an automated evaluation approach of style similarity based on alignment with human evaluations using ChatGPT-4. Our experiments across three resourceful styles show that SETTP requires only 1/20th of the data volume to achieve performance comparable to state-of-the-art methods. In tasks involving scarce data like writing style and role style, SETTP outperforms previous methods by 16.24%.