# Balancing Effect of Training Dataset Distribution of Multiple Styles for Multi-Style Text Transfer

arXiv: https://arxiv.org/abs/2305.15582 · HTML: https://mlsanity.com/paper/2305.15582/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 26 of 26)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Style Transfer in Text: Exploration and Evaluation](https://mlsanity.com/paper/1711.06861/) (2017)
- [Automatically Neutralizing Subjective Bias in Text](https://mlsanity.com/paper/1911.09709/) (2019)
- [Beyond BLEU: Training Neural Machine Translation with Semantic Similarity](https://mlsanity.com/paper/1909.06694/) (2019)
- [EmoBank: Studying the Impact of Annotation Perspective and Representation Format on Dimensional Emotion Analysis](https://mlsanity.com/paper/2205.01996/) (2022)
- [A Probabilistic Formulation of Unsupervised Text Style Transfer](https://mlsanity.com/paper/2002.03912/) (2020)
- [Evaluating Style Transfer for Text](https://mlsanity.com/paper/1904.02295/) (2019)
- [Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Models](https://mlsanity.com/paper/1707.01161/) (2017)
- [Parallel Data Augmentation for Formality Style Transfer](https://mlsanity.com/paper/2005.07522/) (2020)
- [Domain Adaptive Text Style Transfer](https://mlsanity.com/paper/1908.09395/) (2019)
- [Multiple-Attribute Text Style Transfer](https://mlsanity.com/paper/1811.00552/) (2018)
- [Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus](https://mlsanity.com/paper/2010.11578/) (2020)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)
- [Contextualizing Variation in Text Style Transfer Datasets](https://mlsanity.com/paper/2108.07871/) (2021)

## Built on by (most cited first, 2 of 2)

- [Dynamic Multi-Reward Weighting for Multi-Style Controllable Generation](https://mlsanity.com/paper/2402.14146/) (2024)
- [Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding](https://mlsanity.com/paper/2601.17197/) (2026)

## Abstract (arXiv)

Text style transfer is an exciting task within the field of natural language generation that is often plagued by the need for high-quality paired datasets. Furthermore, training a model for multi-attribute text style transfer requires datasets with sufficient support across all combinations of the considered stylistic attributes, adding to the challenges of training a style transfer model. This paper explores the impact of training data input diversity on the quality of the generated text from the multi-style transfer model. We construct a pseudo-parallel dataset by devising heuristics to adjust the style distribution in the training samples. We balance our training dataset using marginal and joint distributions to train our style transfer models. We observe that a balanced dataset produces more effective control effects over multiple styles than an imbalanced or skewed one. Through quantitative analysis, we explore the impact of multiple style distributions in training data on style-transferred output. These findings will better inform the design of style-transfer datasets.