mlsanityData to 7 Oct 2026

Toward Diverse Text Generation with Inverse Reinforcement Learning

arXiv 1804.11258 ↗First posted 30 Apr 2018cs.CL cs.LG stat.MLcited by 41

Not on the map. Why?

Abstract (arXiv)

Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, in this paper, we employ inverse reinforcement learning (IRL) for text generation. Specifically, the IRL framework learns a reward function on training data, and then an optimal policy to maximum the expected total reward. Similar to the adversarial models, the reward and policy function in IRL are optimized alternately. Our method has two advantages: (1) the reward function can produce more dense reward signals. (2) the generation policy, trained by "entropy regularized" policy gradient, encourages to generate more diversified texts. Experiment results demonstrate that our proposed method can generate higher quality texts than the previous methods.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 9; 2023: 13; 2024: 5; 2025: 3; 2026: 1.

41 papers.

Sort:

By area:

  1. 2023Machine Learning for Synthetic Data Generation: A Reviewcited by 118
  2. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  3. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 75
  4. 2023The Next Chapter: A Study of Large Language Models in Storytellingcited by 58
  5. 2024A Dense Reward View on Aligning Text-to-Image Diffusion with Preferencecited by 55
  6. 2021Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speechcited by 42
  7. 2021Efficient (Soft) Q-Learning for Text Generation with Limited Good Datacited by 35
  8. 2021Survey on reinforcement learning for language processingcited by 31
  9. 2023Preference-grounded Token-level Guidance for Language Model Fine-tuningcited by 31
  10. 2023A Review of Deep Learning for Video Captioningcited by 28
  11. 2021Creativity and Machine Learning: A Surveycited by 21
  12. 2022Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Expertscited by 19
29 more
  1. 2022Why is constrained neural language generation particularly challenging?cited by 19
  2. 2023Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Modelscited by 17
  3. 2024What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluationcited by 17
  4. 2022A survey on text generation using generative adversarial networkscited by 16
  5. 2023ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generationcited by 15
  6. 2021DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformercited by 14
  7. 2023SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtrackingcited by 13
  8. 2025Segmenting Text and Learning Their Rewards for Improved RLHF in Language Modelcited by 12
  9. 2022Teacher Forcing Recovers Reward Functions for Text Generationcited by 11
  10. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  11. 2022Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlookcited by 10
  12. 2023Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challengescited by 10
  13. 2023Token Imbalance Adaptation for Radiology Report Generationcited by 9
  14. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  15. 2022Efficient Reinforcement Learning for Unsupervised Controlled Text Generationcited by 7
  16. 2023Autoregressive Modeling with Lookahead Attentioncited by 7
  17. 2022Inverse Reinforcement Learning for Text Summarizationcited by 5
  18. 2024It Takes Two: On the Seamlessness between Reward and Policy Model in RLHFcited by 5
  19. 2021Sentence-Permuted Paragraph Generationcited by 4
  20. 2022Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attentioncited by 3
  21. 2025Leveraging Partial SMILES Validation Scheme for Enhanced Drug Design in Reinforcement Learning Frameworkscited by 3
  22. 2021Mapping Language to Programs using Multiple Reward Components with Inverse Reinforcement Learningcited by 2
  23. 2023Predicting Visit Cost of Obstructive Sleep Apnea using Electronic Healthcare Records with Transformercited by 1
  24. 2025Unbiased Sliced Wasserstein Kernels for High-Quality Audio Captioningcited by 1
  25. 2021CatVRNN: Generating Category Texts via Multi-task Learningcited by 0
  26. 2021Reinforcement Learning for Few-Shot Text Generation Adaptationcited by 0
  27. 2021Molecular Attributes Transfer from Non-Parallel Datacited by 0
  28. 2024TCR-GPT: Integrating Autoregressive Model and Reinforcement Learning for T-Cell Receptor Repertoires Generationcited by 0
  29. 2026Gacha Decoding: Eliciting Diverse Generations Through Instruction Followingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020TextGAIL: Generative Adversarial Imitation Learning for Text Generation
  2. 2020Text Generation by Learning from Demonstrations
  3. 2024Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generation
  4. 2018Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning
  5. 2025Towards Cost-Effective Reward Guided Text Generation
  6. 2024A Critical Look At Tokenwise Reward-Guided Text Generation
  7. 2023Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model
  8. 2025LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning