# CIE: Controlling Language Model Text Generations Using Continuous Signals

arXiv: https://arxiv.org/abs/2505.13448 · HTML: https://mlsanity.com/paper/2505.13448/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 17 of 17)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting](https://mlsanity.com/paper/2305.15685/) (2023)
- [Following Length Constraints in Instructions](https://mlsanity.com/paper/2406.17744/) (2024)
- [A Language Model's Guide Through Latent Space](https://mlsanity.com/paper/2402.14433/) (2024)
- [Number Cookbook: Number Understanding of Language Models and How to Improve It](https://mlsanity.com/paper/2411.03766/) (2024)
- [What's the Magic Word? A Control Theory of LLM Prompting](https://mlsanity.com/paper/2310.04444/) (2023)
- [Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance](https://mlsanity.com/paper/2402.14531/) (2024)
- [Mixture of Soft Prompts for Controllable Data Generation](https://mlsanity.com/paper/2303.01580/) (2023)
- [SarcasmBench: Towards Evaluating Large Language Models on Sarcasm Understanding](https://mlsanity.com/paper/2408.11319/) (2024)
- [Human-aligned Chess with a Bit of Search](https://mlsanity.com/paper/2410.03893/) (2024)
- [RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation](https://mlsanity.com/paper/2305.17131/) (2023)

## Built on by (most cited first, 1 of 1)

- [Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation](https://mlsanity.com/paper/2609.01246/) (2026)

## Abstract (arXiv)

Aligning language models (LMs) with user intent is becoming increasingly relevant to enhance user experience. This calls for designing methods that can allow users to control the properties of the language that LMs generate, for example, controlling the length of the generation or the complexity of the language that gets chosen. Most existing work attempts to integrate users' control by conditioning LM generations on natural language prompts or discrete control signals, which are often brittle and hard to scale. In this work, we are interested in continuous control signals, ones that exist along a spectrum that can't easily be captured in a natural language prompt or via existing techniques in conditional generation. Through a case study in controlling the precise response-length of generations, we demonstrate how an LM can be finetuned to expect a control vector that is interpolated between a "low" and a "high" token embedding. Our method more reliably exerts response-length control than in-context learning methods or fine-tuning methods that represent the control signal as a discrete signal.