mlsanityData to 14 Oct 2026

Why is constrained neural language generation particularly challenging?

arXiv 2206.05395 ↗First posted 11 Jun 2022cs.CL cs.AIcited by 19

Not on the map. Why?

Abstract (arXiv)

Recent advances in deep neural language models combined with the capacity of large scale datasets have accelerated the development of natural language generation systems that produce fluent and coherent texts (to various degrees of success) in a multitude of tasks and application contexts. However, controlling the output of these models for desired user and task needs is still an open challenge. This is crucial not only to customizing the content and style of the generated language, but also to their safe and reliable deployment in the real world. We present an extensive survey on the emerging topic of constrained neural language generation in which we formally define and categorize the problems of natural language generation by distinguishing between conditions and constraints (the latter being testable conditions on the output text instead of the input), present constrained text generation tasks, and review existing methods and evaluation metrics for constrained text generation. Our aim is to highlight recent progress and trends in this emerging field, informing on the most promising directions and limitations towards advancing the state-of-the-art of constrained neural language generation research.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 130 of its 177 references that are in the map. First posted on arXiv 11 Jun 2022.

Builds on

130 of 177 references.

References · 130

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  7. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  8. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  9. 2018Hierarchical Neural Story Generationcited by 1,193
  10. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  11. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  12. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
38 more
  1. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  2. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  3. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  4. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 722
  5. 2015Generating Sentences from a Continuous Spacecited by 612
  6. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  7. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  8. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  9. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  10. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 418
  11. 2018Assessing Generative Models via Precision and Recallcited by 406
  12. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 396
  13. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  14. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  15. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 365
  16. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  17. 2018Generating Wikipedia by Summarizing Long Sequencescited by 335
  18. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  19. 2015A note on the evaluation of generative modelscited by 292
  20. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  21. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 286
  22. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 284
  23. 2017Toward Controlled Generation of Textcited by 272
  24. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  25. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 248
  26. 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 246
  27. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 237
  28. 2017Challenges in Data-to-Document Generationcited by 226
  29. 2018Pros and Cons of GAN Evaluation Measurescited by 224
  30. 2016A Persona-Based Neural Conversation Modelcited by 223
  31. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 211
  32. 2015Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Modelscited by 207
  33. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  34. 2016Enhanced LSTM for Natural Language Inferencecited by 200
  35. 2019Levenshtein Transformercited by 199
  36. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 195
  37. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
  38. 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 182

50 most cited of 130 All 130 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 7; 2025: 4; 2026: 2.

19 papers.

Sort:

By area:

  1. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
  2. 2023PREADD: Prefix-Adaptive Decoding for Controlled Text Generationcited by 29
  3. 2022ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Modelscited by 25
  4. 2023Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challengescited by 22
  5. 2025WildIFEval: Instruction Following in the Wildcited by 11
  6. 2024Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generationcited by 10
  7. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 10
  8. 2024Combining Constraint Programming Reasoning with Large Language Model Predictionscited by 8
  9. 2022Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Reviewcited by 3
  10. 2024Intertwining CP and NLP: The Generation of Unreasonably Constrained Sentencescited by 3
  11. 2026Constraint Tax in Open-Weight LLMs: An Empirical Study of Tool Calling Suppression Under Structured Output Constraintscited by 3
  12. 2024ExpressivityBench: Can LLMs Communicate Implicitly?cited by 2
7 more
  1. 2025Large Language Model Meets Constraint Propagationcited by 2
  2. 2026Deconstructing Instruction-Following: A New Benchmark for Granular Evaluation of Large Language Model Instruction Compliance Abilitiescited by 2
  3. 2023Advancing Perception in Artificial Intelligence through Principles of Cognitive Sciencecited by 1
  4. 2025An End-to-End System for Culturally-Attuned Driving Feedback using a Dual-Component NLG Enginecited by 1
  5. 2022IFDID: Information Filter upon Diversity-Improved Decoding for Diversity-Faithfulness Tradeoff in NLGcited by 0
  6. 2024BIPro: Zero-shot Chinese Poem Generation via Block Inverse Prompting Constrained Generation Frameworkcited by 0
  7. 2025Orthographic Constraint Satisfaction and Human Difficulty Alignment in Large Language Modelscited by 0