mlsanityData to 7 Oct 2026

A Distributional Approach to Controlled Text Generation

arXiv 2012.11635 ↗First posted 21 Dec 2020cs.CL cs.AI cs.LGcited by 117

Not on the map. Why?

Cited by 33 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a Distributional Approach for addressing Controlled Text Generation from pre-trained Language Models (LMs). This approach permits to specify, in a single formal framework, both "pointwise" and "distributional" constraints over the target LM -- to our knowledge, the first model with such generality -- while minimizing KL divergence from the initial LM distribution. The optimal target distribution is then uniquely determined as an explicit EBM (Energy-Based Model) representation. From that optimal representation we then train a target controlled Autoregressive LM through an adaptive distributional variant of Policy Gradient. We conduct a first set of experiments over pointwise constraints showing the advantages of our approach over a set of baselines, in terms of obtaining a controlled LM balancing constraint satisfaction with divergence from the initial LM. We then perform experiments over distributional constraints, a unique feature of our approach, demonstrating its potential as a remedy to the problem of Bias in Language Models. Through an ablation study, we show the effectiveness of our adaptive technique for obtaining faster convergence. (Code available at https://github.com/naver/gdc)

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 24; 2023: 24; 2024: 28; 2025: 18; 2026: 12.

117 papers.

Sort:

By area:

  1. 2021Ethical and social risks of harm from Language Modelscited by 957
  2. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  3. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  4. 2023Pretraining Language Models with Human Preferencescited by 245
  5. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  6. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  7. 2023Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraintscited by 174
  8. 2021Challenges in Detoxifying Language Modelscited by 169
  9. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 159
  10. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 136
  11. 2024Token-level Direct Preference Optimizationcited by 135
  12. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
38 more
  1. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  2. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  3. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  4. 2024Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMscited by 103
  5. 2023Confronting Reward Model Overoptimization with Constrained RLHFcited by 96
  6. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 85
  7. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  8. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 79
  9. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  10. 2022Mix and Match: Learning-free Controllable Text Generation using Energy Language Modelscited by 66
  11. 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
  12. 2022Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Modelscited by 49
  13. 2023Learning to Generate Better Than Your LLMcited by 49
  14. 2022Controllable Text Generation with Neurally-Decomposed Oraclecited by 41
  15. 2022Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learningcited by 36
  16. 2023GRACE: Discriminator-Guided Chain-of-Thought Reasoningcited by 35
  17. 2022A Distributional Lens for Multi-Aspect Controllable Text Generationcited by 33
  18. 2024From Distributional to Overton Pluralism: Investigating Large Language Model Alignmentcited by 33
  19. 2021Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlinescited by 32
  20. 2021Controlling Conditional Language Models without Catastrophic Forgettingcited by 32
  21. 2023Preference-grounded Token-level Guidance for Language Model Fine-tuningcited by 31
  22. 2024Preference Tuning For Toxicity Mitigation Generalizes Across Languagescited by 31
  23. 2022DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generationcited by 30
  24. 2022Reward Gaming in Conditional Text Generationcited by 29
  25. 2022Composable Text Controls in Latent Space with ODEscited by 28
  26. 2022Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Modelscited by 27
  27. 2022Controllable Text Generation via Probability Density Estimation in the Latent Spacecited by 27
  28. 2024Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learningcited by 27
  29. 2023BOLT: Fast Energy-based Controlled Text Generation with Tunable Biasescited by 24
  30. 2021Unsupervised and Distributional Detection of Machine-Generated Textcited by 23
  31. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  32. 2023Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHFcited by 20
  33. 2022Why is constrained neural language generation particularly challenging?cited by 19
  34. 2021Refocusing on Relevance: Personalization in NLGcited by 18
  35. 2023Interpretable Math Word Problem Solution Generation Via Step-by-step Planningcited by 17
  36. 2023Flesch or Fumble? Evaluating Readability Standard Alignment of Instruction-Tuned Language Modelscited by 17
  37. 2024On the Limitations of Compute Thresholds as a Governance Strategycited by 17
  38. 2025BLEUBERI: BLEU is a surprisingly effective reward for instruction followingcited by 17

50 most cited of 117 All 117 (JSON)