mlsanityData to 14 Oct 2026

BOLT: Fast Energy-based Controlled Text Generation with Tunable Biases

arXiv 2305.12018 ↗First posted 19 May 2023cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Energy-based models (EBMs) have gained popularity for controlled text generation due to their high applicability to a wide range of constraints. However, sampling from EBMs is non-trivial, as it often requires a large number of iterations to converge to plausible text, which slows down the decoding process and makes it less practical for real-world applications. In this work, we propose BOLT, which relies on tunable biases to directly adjust the language model's output logits. Unlike prior work, BOLT maintains the generator's autoregressive nature to assert a strong control on token-wise conditional dependencies and overall fluency, and thus converges faster. When compared with state-of-the-arts on controlled generation tasks using both soft constraints (e.g., sentiment control) and hard constraints (e.g., keyword-guided topic control), BOLT demonstrates significantly improved efficiency and fluency. On sentiment control, BOLT is 7x faster than competitive baselines, and more fluent in 74.4% of the evaluation samples according to human judges.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 25 of its 37 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

25 of 37 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  5. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  6. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  7. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  8. 2016Self-critical Sequence Training for Image Captioningcited by 547
  9. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  10. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  11. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  12. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
13 more
  1. 2020A Distributional Approach to Controlled Text Generationcited by 119
  2. 2016Improved Image Captioning via Policy Gradient optimization of SPIDErcited by 113
  3. 2021Jigsaw: Large Language Models meet Program Synthesiscited by 103
  4. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  5. 2022Mix and Match: Learning-free Controllable Text Generation using Energy Language Modelscited by 66
  6. 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
  7. 2020Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generationcited by 47
  8. 2021Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastingscited by 37
  9. 2022DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generationcited by 30
  10. 2022Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generationcited by 22
  11. 2019Controlled Text Generation for Data Augmentation in Intelligent Artificial Agentscited by 21
  12. 2021Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generationcited by 17
  13. 2017Differentiable lower bound for expected BLEU scorecited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 6; 2026: 7.

24 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  2. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
  3. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 45
  4. 2025FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steeringcited by 40
  5. 2023GRACE: Discriminator-Guided Chain-of-Thought Reasoningcited by 35
  6. 2024DeepEdit: Knowledge Editing as Decoding with Constraintscited by 33
  7. 2024Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practicescited by 20
  8. 2024Controllable Text Generation in the Instruction-Tuning Eracited by 7
  9. 2024DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questionscited by 7
  10. 2025BiasFilter: An Inference-Time Debiasing Framework for Large Language Modelscited by 6
  11. 2025Automated Knowledge Component Generation for Interpretable Knowledge Tracing in Coding Problemscited by 5
  12. 2026nabla-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Spacecited by 5
12 more
  1. 2025Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMscited by 4
  2. 2024LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localizationcited by 2
  3. 2025Palette of Language Models: A Solver for Controlled Text Generationcited by 2
  4. 2025Precise Attribute Intensity Control in Large Language Models via Targeted Representation Editingcited by 2
  5. 2024Linguistically-Controlled Paraphrase Generationcited by 1
  6. 2026Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMscited by 1
  7. 2026DRIV-EX: Counterfactual Explanations for Driving LLMscited by 1
  8. 2024LingGen: Scalable Multi-Attribute Linguistic Control via Power-Law Maskingcited by 0
  9. 2026Revisiting Transformer Layer Parameterization Through Causal Energy Minimizationcited by 0
  10. 2026Gradient-Guided Reward Optimization for Inference-time Alignmentcited by 0
  11. 2026When Debiasing Backfires: Counterintuitive Side Effects of Preprocessing-Based Stereotype Mitigationcited by 0
  12. 2026Mitigating Bias in Large Vision-Language Models via Counterfactual Ensemble Decodingcited by 0