mlsanityData to 14 Oct 2026

NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints

arXiv 2010.12884 ↗First posted 24 Oct 2020cs.CLcited by 149

Not on the map. Why?

Abstract (arXiv)

Conditional text generation often requires lexical constraints, i.e., which words should or shouldn't be included in the output text. While the dominant recipe for conditional text generation has been large-scale pretrained language models that are finetuned on the task-specific training data, such models do not learn to follow the underlying constraints reliably, even when supervised with large amounts of task-specific examples. We propose NeuroLogic Decoding, a simple yet effective algorithm that enables neural language models -- supervised or not -- to generate fluent text while satisfying complex lexical constraints. Our approach is powerful yet efficient. It handles any set of lexical constraints that is expressible under predicate logic, while its asymptotic runtime is equivalent to conventional beam search. Empirical results on four benchmarks show that NeuroLogic Decoding outperforms previous approaches, including algorithms that handle a subset of our constraints. Moreover, we find that unsupervised models with NeuroLogic Decoding often outperform supervised models with conventional decoding, even when the latter is based on considerably larger networks. Our results suggest the limit of large-scale neural networks for fine-grained controllable generation and the promise of inference-time algorithms.

Key facts. Cited by 149 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 26; 2023: 42; 2024: 28; 2025: 17; 2026: 27.

149 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  2. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  3. 2022Generating Sequences by Learning to Self-Correctcited by 277
  4. 2022A Contrastive Framework for Neural Text Generationcited by 225
  5. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  6. 2024A Roadmap to Pluralistic Alignmentcited by 184
  7. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
  8. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  9. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  10. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
  11. 2024ARGS: Alignment as Reward-Guided Searchcited by 103
  12. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Differencecited by 88
38 more
  1. 2023Controlled Text Generation with Natural Language Instructionscited by 87
  2. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  3. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  4. 2023Evaluating Large Language Models on Controlled Generation Taskscited by 73
  5. 2022NaturalProver: Grounded Mathematical Proof Generation with Language Modelscited by 72
  6. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  7. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  8. 2023COLLIE: Systematic Construction of Constrained Text Generation Taskscited by 70
  9. 2023Tractable Control for Autoregressive Language Generationcited by 65
  10. 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
  11. 2023Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decodingcited by 60
  12. 2023Harnessing the Power of Large Language Models for Natural Language to First-Order Logic Translationcited by 59
  13. 2024DeAL: Decoding-time Alignment for Large Language Modelscited by 55
  14. 2024Grammar-Aligned Decodingcited by 54
  15. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 45
  16. 2022LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restrictioncited by 43
  17. 2024To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Modelscited by 43
  18. 2022Controllable Text Generation with Neurally-Decomposed Oraclecited by 41
  19. 2025FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steeringcited by 40
  20. 2024Constrained Decoding for Secure Code Generationcited by 37
  21. 2022Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computingcited by 35
  22. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  23. 2024DeepEdit: Knowledge Editing as Decoding with Constraintscited by 33
  24. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  25. 2024Adaptable Logical Control for Large Language Modelscited by 30
  26. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29
  27. 2022Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processingcited by 23
  28. 2022NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentationcited by 23
  29. 2023A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraintscited by 22
  30. 2022I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitationcited by 21
  31. 2025Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Samplingcited by 21
  32. 2022Why is constrained neural language generation particularly challenging?cited by 19
  33. 2024DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inferencecited by 19
  34. 2022Controllable Text Generation with Language Constraintscited by 18
  35. 2024StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elementscited by 18
  36. 2025Constrained Discrete Diffusioncited by 17
  37. 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectivescited by 16
  38. 2022BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examplescited by 15

50 most cited of 149 All 149 (JSON)