# Palette of Language Models: A Solver for Controlled Text Generation

arXiv: https://arxiv.org/abs/2503.11182 · HTML: https://mlsanity.com/paper/2503.11182/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 28 of 28)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics](https://mlsanity.com/paper/2104.13346/) (2021)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Quark: Controllable Text Generation with Reinforced Unlearning](https://mlsanity.com/paper/2205.13636/) (2022)
- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](https://mlsanity.com/paper/2202.11705/) (2022)
- [How well do Large Language Models perform in Arithmetic tasks?](https://mlsanity.com/paper/2304.02015/) (2023)
- [Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text](https://mlsanity.com/paper/2012.04698/) (2020)
- [Controllable Natural Language Generation with Contrastive Prefixes](https://mlsanity.com/paper/2202.13257/) (2022)
- [Controlled Text Generation as Continuous Optimization with Multiple Constraints](https://mlsanity.com/paper/2108.01850/) (2021)
- [Mix and Match: Learning-free Controllable Text Generation using Energy Language Models](https://mlsanity.com/paper/2203.13299/) (2022)
- [Gradient-Based Constrained Sampling from Language Models](https://mlsanity.com/paper/2205.12558/) (2022)
- [Controlled Text Generation via Language Model Arithmetic](https://mlsanity.com/paper/2311.14479/) (2023)
- [PREADD: Prefix-Adaptive Decoding for Controlled Text Generation](https://mlsanity.com/paper/2307.03214/) (2023)
- [Raiders of the Lost Kek: 3.5 Years of Augmented 4chan Posts from the Politically Incorrect Board](https://mlsanity.com/paper/2001.07487/) (2020)
- [Control Prefixes for Parameter-Efficient Text Generation](https://mlsanity.com/paper/2110.08329/) (2021)
- [BOLT: Fast Energy-based Controlled Text Generation with Tunable Biases](https://mlsanity.com/paper/2305.12018/) (2023)
- [Learning to Transfer Prompts for Text Generation](https://mlsanity.com/paper/2205.01543/) (2022)
- [POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training](https://mlsanity.com/paper/2005.00558/) (2020)
- [Focused Prefix Tuning for Controllable Text Generation](https://mlsanity.com/paper/2306.00369/) (2023)

## Built on by (most cited first, 2 of 2)

- [C3TG: Conflict-aware, Composite, and Collaborative Controlled Text Generation](https://mlsanity.com/paper/2511.09292/) (2025)
- [LaPA2: Length-Aware Prefix and Prompt Attention Augmentation for Long-Form Controllable Text Generation](https://mlsanity.com/paper/2508.04047/) (2025)

## Abstract (arXiv)

Recent advancements in large language models have revolutionized text generation with their remarkable capabilities. These models can produce controlled texts that closely adhere to specific requirements when prompted appropriately. However, designing an optimal prompt to control multiple attributes simultaneously can be challenging. A common approach is to linearly combine single-attribute models, but this strategy often overlooks attribute overlaps and can lead to conflicts. Therefore, we propose a novel combination strategy inspired by the Law of Total Probability and Conditional Mutual Information Minimization on generative language models. This method has been adapted for single-attribute control scenario and is termed the Palette of Language Models due to its theoretical linkage between attribute strength and generation style, akin to blending colors on an artist's palette. Moreover, positive correlation and attribute enhancement are advanced as theoretical properties to guide a rational combination strategy design. We conduct experiments on both single control and multiple control settings, and achieve surpassing results.