# Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs

arXiv: https://arxiv.org/abs/2402.11218 · HTML: https://mlsanity.com/paper/2402.11218/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 17 Feb 2024.

## Builds on (most cited first, 29 of 29)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning](https://mlsanity.com/paper/2312.01552/) (2023)
- [RAIN: Your Language Models Can Align Themselves without Finetuning](https://mlsanity.com/paper/2309.07124/) (2023)
- [DART: Open-Domain Structured Data Record to Text Generation](https://mlsanity.com/paper/2007.02871/) (2020)
- [Controlled Text Generation with Natural Language Instructions](https://mlsanity.com/paper/2304.14293/) (2023)
- [Controllable Natural Language Generation with Contrastive Prefixes](https://mlsanity.com/paper/2202.13257/) (2022)
- [Self-Detoxifying Language Models via Toxification Reversal](https://mlsanity.com/paper/2310.09573/) (2023)
- [Critic-Guided Decoding for Controlled Text Generation](https://mlsanity.com/paper/2212.10938/) (2022)
- [DIRECTOR: Generator-Classifiers For Supervised Language Modeling](https://mlsanity.com/paper/2206.07694/) (2022)
- [DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation](https://mlsanity.com/paper/2210.09551/) (2022)
- [CounterGeDi: A controllable approach to generate polite, detoxified and emotional counterspeech](https://mlsanity.com/paper/2205.04304/) (2022)
- [PREADD: Prefix-Adaptive Decoding for Controlled Text Generation](https://mlsanity.com/paper/2307.03214/) (2023)
- [Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models](https://mlsanity.com/paper/2310.07589/) (2023)
- [Classifiers are Better Experts for Controllable Text Generation](https://mlsanity.com/paper/2205.07276/) (2022)
- [Air-Decoding: Attribute Distribution Reconstruction for Decoding-Time Controllable Text Generation](https://mlsanity.com/paper/2310.14892/) (2023)
- [CAT-LLM: Style-enhanced Large Language Models with Text Style Definition for Chinese Article-style Transfer](https://mlsanity.com/paper/2401.05707/) (2024)

## Built on by (most cited first, 16 of 16)

- [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024)
- [Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM](https://mlsanity.com/paper/2508.05775/) (2025)
- [Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.03098/) (2025)
- [Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models](https://mlsanity.com/paper/2405.12939/) (2024)
- [JAM: Controllable and Responsible Text Generation via Causal Reasoning and Latent Vector Manipulation](https://mlsanity.com/paper/2502.20684/) (2025)
- [PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts](https://mlsanity.com/paper/2503.06706/) (2025)
- [C3TG: Conflict-aware, Composite, and Collaborative Controlled Text Generation](https://mlsanity.com/paper/2511.09292/) (2025)
- [LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localization](https://mlsanity.com/paper/2407.00740/) (2024)
- [Controlling Distributional Bias in Multi-Round LLM Generation via KL-Optimized Fine-Tuning](https://mlsanity.com/paper/2604.05756/) (2026)
- [Speak & Spell: LLM-Driven Controllable Phonetic Error Augmentation for Robust Dialogue State Tracking](https://mlsanity.com/paper/2409.06263/) (2024)
- [Stance-Driven Multimodal Controlled Statement Generation: New Dataset and Task](https://mlsanity.com/paper/2504.03295/) (2025)
- [LaPA2: Length-Aware Prefix and Prompt Attention Augmentation for Long-Form Controllable Text Generation](https://mlsanity.com/paper/2508.04047/) (2025)
- [Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructure](https://mlsanity.com/paper/2604.25580/) (2026)
- [AgentCTG: Harnessing Multi-Agent Collaboration for Fine-Grained Precise Control in Text Generation](https://mlsanity.com/paper/2509.13677/) (2025)
- [Generating Reading Comprehension Exercises with Large Language Models for Educational Applications](https://mlsanity.com/paper/2511.18860/) (2025)
- [A Cross-lingual Comparison of Human and Classification Model Entrainment Behavior in Code-switched Speech Settings](https://mlsanity.com/paper/2607.25202/) (2026)

## Abstract (arXiv)

Controlled Text Generation (CTG) aims to produce texts that exhibit specific desired attributes. In this study, we introduce a pluggable CTG framework for Large Language Models (LLMs) named Dynamic Attribute Graphs-based controlled text generation (DATG). This framework utilizes an attribute scorer to evaluate the attributes of sentences generated by LLMs and constructs dynamic attribute graphs. DATG modulates the occurrence of key attribute words and key anti-attribute words, achieving effective attribute control without compromising the original capabilities of the model. We conduct experiments across four datasets in two tasks: toxicity mitigation and sentiment transformation, employing five LLMs as foundational models. Our findings highlight a remarkable enhancement in control accuracy, achieving a peak improvement of 19.29% over baseline methods in the most favorable task across four datasets. Additionally, we observe a significant decrease in perplexity, markedly improving text fluency.