# Incorporating Stylistic Lexical Preferences in Generative Language Models

arXiv: https://arxiv.org/abs/2010.11553 · HTML: https://mlsanity.com/paper/2010.11553/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 22 Oct 2020.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [Stabilizing Transformers for Reinforcement Learning](https://mlsanity.com/paper/1910.06764/) (2019)
- [Paraphrase Generation with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.00279/) (2017)
- [Adapting Language Models for Non-Parallel Author-Stylized Rewriting](https://mlsanity.com/paper/1909.09962/) (2019)
- [Complexity of Word Collocation Networks: A Preliminary Structural Analysis](https://mlsanity.com/paper/1310.5111/) (2013)
- [A Lexical, Syntactic, and Semantic Perspective for Understanding Style in Text](https://mlsanity.com/paper/1909.08349/) (2019)

## Built on by (most cited first, 2 of 2)

- [DRAG: Director-Generator Language Modelling Framework for Non-Parallel Author Stylized Rewriting](https://mlsanity.com/paper/2101.11836/) (2021)
- [Improvement in Machine Translation with Generative Adversarial Networks](https://mlsanity.com/paper/2111.15166/) (2021)

## Abstract (arXiv)

While recent advances in language modeling have resulted in powerful generation models, their generation style remains implicitly dependent on the training data and can not emulate a specific target style. Leveraging the generative capabilities of a transformer-based language models, we present an approach to induce certain target-author attributes by incorporating continuous multi-dimensional lexical preferences of an author into generative language models. We introduce rewarding strategies in a reinforcement learning framework that encourages the use of words across multiple categorical dimensions, to varying extents. Our experiments demonstrate that the proposed approach can generate text that distinctively aligns with a given target author's lexical style. We conduct quantitative and qualitative comparisons with competitive and relevant baselines to illustrate the benefits of the proposed approach.