mlsanityData to 7 Oct 2026

End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient

arXiv 1712.02838 ↗First posted 7 Dec 2017cs.AI cs.CL cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Learning a goal-oriented dialog policy is generally performed offline with supervised learning algorithms or online with reinforcement learning (RL). Additionally, as companies accumulate massive quantities of dialog transcripts between customers and trained human agents, encoder-decoder methods have gained popularity as agent utterances can be directly treated as supervision without the need for utterance-level annotations. However, one potential drawback of such approaches is that they myopically generate the next agent utterance without regard for dialog-level considerations. To resolve this concern, this paper describes an offline RL method for learning from unannotated corpora that can optimize a goal-oriented policy at both the utterance and dialog level. We introduce a novel reward function and use both on-policy and off-policy policy gradient to learn a policy offline without requiring online user interaction or an explicit state space definition.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2020: 3; 2021: 5; 2022: 4; 2023: 8; 2024: 2; 2025: 1.

23 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  3. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  4. 2021Efficient (Soft) Q-Learning for Text Generation with Limited Good Datacited by 35
  5. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  6. 2023Vanishing Gradients in Reinforcement Finetuning of Language Modelscited by 26
  7. 2021Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RLcited by 24
  8. 2024Prompt-Based Length Controlled Generation with Multiple Control Typescited by 19
  9. 2023Prompt-Based Length Controlled Generation with Reinforcement Learningcited by 14
  10. 2023STEER: Unified Style Transfer with Expert Reinforcementcited by 14
  11. 2020Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Taskscited by 12
  12. 2023Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challengescited by 10
11 more
  1. 2025Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimizationcited by 10
  2. 2022KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learningcited by 9
  3. 2023On the Effectiveness of Offline RL for Dialogue Response Generationcited by 6
  4. 2021Amortized Noisy Channel Neural Machine Translationcited by 4
  5. 2023On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samplescited by 4
  6. 2021Reducing Conservativeness Oriented Offline Reinforcement Learningcited by 3
  7. 2023Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compressioncited by 3
  8. 2021A Review of Dialogue Systems: From Trained Monkeys to Stochastic Parrotscited by 1
  9. 2022Reinforcement Learning of Multi-Domain Dialog Policies Via Action Embeddingscited by 0
  10. 2022Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecturecited by 0
  11. 2024Robust Offline Reinforcement Learning for Non-Markovian Decision Processescited by 0