# Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer

arXiv: https://arxiv.org/abs/2305.05945 · HTML: https://mlsanity.com/paper/2305.05945/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Style Transfer in Text: Exploration and Evaluation](https://mlsanity.com/paper/1711.06861/) (2017)
- [Disentangled Representation Learning for Non-Parallel Text Style Transfer](https://mlsanity.com/paper/1808.04339/) (2018)
- [Style Transfer Through Back-Translation](https://mlsanity.com/paper/1804.09000/) (2018)
- [Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model](https://mlsanity.com/paper/1912.09637/) (2019)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)
- [Politeness Transfer: A Tag and Generate Approach](https://mlsanity.com/paper/2004.14257/) (2020)
- [A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer](https://mlsanity.com/paper/1905.10060/) (2019)
- [Content preserving text generation with attribute controls](https://mlsanity.com/paper/1811.01135/) (2018)
- [StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer](https://mlsanity.com/paper/2104.05196/) (2021)
- [Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter](https://mlsanity.com/paper/2105.07148/) (2021)
- [Adversarial Text Generation via Feature-Mover's Distance](https://mlsanity.com/paper/1809.06297/) (2018)
- [Integrating Graph Contextualized Knowledge into Pre-trained Language Models](https://mlsanity.com/paper/1912.00147/) (2019)
- [Domain Adaptive Text Style Transfer](https://mlsanity.com/paper/1908.09395/) (2019)
- [Adapting Language Models for Non-Parallel Author-Stylized Rewriting](https://mlsanity.com/paper/1909.09962/) (2019)
- [Style Transfer as Unsupervised Machine Translation](https://mlsanity.com/paper/1808.07894/) (2018)
- [Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus](https://mlsanity.com/paper/2010.11578/) (2020)
- [Are Current Task-oriented Dialogue Systems Able to Satisfy Impolite Users?](https://mlsanity.com/paper/2210.12942/) (2022)

## Built on by (most cited first, 2 of 2)

- [Personalized Text Generation with Fine-Grained Linguistic Control](https://mlsanity.com/paper/2402.04914/) (2024)
- [Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai](https://mlsanity.com/paper/2411.15484/) (2024)

## Abstract (arXiv)

Adapting a large language model for multiple-attribute text style transfer via fine-tuning can be challenging due to the significant amount of computational resources and labeled data required for the specific task. In this paper, we address this challenge by introducing AdapterTST, a framework that freezes the pre-trained model's original parameters and enables the development of a multiple-attribute text style transfer model. Using BART as the backbone model, Adapter-TST utilizes different neural adapters to capture different attribute information, like a plug-in connected to BART. Our method allows control over multiple attributes, like sentiment, tense, voice, etc., and configures the adapters' architecture to generate multiple outputs respected to attributes or compositional editing on the same sentence. We evaluate the proposed model on both traditional sentiment transfer and multiple-attribute transfer tasks. The experiment results demonstrate that Adapter-TST outperforms all the state-of-the-art baselines with significantly lesser computational resources. We have also empirically shown that each adapter is able to capture specific stylistic attributes effectively and can be configured to perform compositional editing.