mlsanityData to 7 Oct 2026

Constrained Text Generation with Global Guidance -- Case Study on CommonGen

arXiv 2103.07170 ↗First posted 12 Mar 2021cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

This paper studies constrained text generation, which is to generate sentences under certain pre-conditions. We focus on CommonGen, the task of generating text based on a set of concepts, as a representative task of constrained text generation. Traditional methods mainly rely on supervised training to maximize the likelihood of target sentences.However, global constraints such as common sense and coverage cannot be incorporated into the likelihood objective of the autoregressive decoding process. In this paper, we consider using reinforcement learning to address the limitation, measuring global constraints including fluency, common sense and concept coverage with a comprehensive score, which serves as the reward for reinforcement learning. Besides, we design a guided decoding method at the word, fragment and sentence levels. Experiments demonstrate that our method significantly increases the concept coverage and outperforms existing models in various automatic evaluations.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 12 Mar 2021.

Builds on

33 of 50 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2018Deep contextualized word representationscited by 2,285
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  10. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  11. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  12. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
21 more
  1. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  2. 2019Text Summarization with Pretrained Encoderscited by 553
  3. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  4. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  5. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  6. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  7. 2020UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Trainingcited by 204
  8. 2018Learning to Write with Cooperative Discriminatorscited by 119
  9. 2020A Knowledge-Enhanced Pretraining Model for Commonsense Story Generationcited by 116
  10. 2018Data-to-Text Generation with Content Selection and Planningcited by 106
  11. 2018A Study of Reinforcement Learning for Neural Machine Translationcited by 74
  12. 2018A Graph-to-Sequence Model for AMR-to-Text Generationcited by 56
  13. 2019Topic-Guided Variational Autoencoders for Text Generationcited by 44
  14. 2019Does It Make Sense? And Why? A Pilot Study for Sense Making and Explanationcited by 43
  15. 2020SemEval-2020 Task 4: Commonsense Validation and Explanationcited by 42
  16. 2020GPT-too: A language-model-first approach for AMR-to-text generationcited by 38
  17. 2021Structural Information Preserving for Graph-to-Text Generationcited by 24
  18. 2020Adversarial Attack and Defense of Structured Prediction Modelscited by 20
  19. 2019Structural Neural Encoders for AMR-to-text Generationcited by 19
  20. 2019CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoningcited by 12
  21. 2019An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 3; 2025: 2; 2026: 1.

10 papers.

  1. 2022Controllable Text Generation with Language Constraintscited by 18
  2. 2023Most Language Models can be Poets too: An AI Writing Assistant and Constrained Text Generation Studiocited by 9
  3. 2024Combining Constraint Programming Reasoning with Large Language Model Predictionscited by 8
  4. 2023Constraints First: A New MDD-based Model to Generate Sentences Under Constraintscited by 4
  5. 2024Markov Constraint as Large Language Model Surrogatecited by 3
  6. 2024Intertwining CP and NLP: The Generation of Unreasonably Constrained Sentencescited by 3
  7. 2022Lexical Complexity Controlled Sentence Generationcited by 2
  8. 2025Estimating Commonsense Plausibility through Semantic Shiftscited by 1
  9. 2025Gatsby Without the 'E': Crafting Lipograms with LLMscited by 0
  10. 2026The Maskability Index: Predicting Task-Objective Alignment in Pretrained Language Modelscited by 0