# Hierarchical Text Generation and Planning for Strategic Dialogue

arXiv: https://arxiv.org/abs/1712.05846 · HTML: https://mlsanity.com/paper/1712.05846/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2017.

## Built on by (most cited first, 23 of 23)

- [The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text](https://mlsanity.com/paper/2311.09807/) (2023)
- [Modeling Strong and Human-Like Gameplay with KL-Regularized Search](https://mlsanity.com/paper/2112.07544/) (2021)
- [Benchmarking Linguistic Diversity of Large Language Models](https://mlsanity.com/paper/2412.10271/) (2024)
- [CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning](https://mlsanity.com/paper/2204.08426/) (2022)
- [Reason first, then respond: Modular Generation for Knowledge-infused Dialogue](https://mlsanity.com/paper/2111.05204/) (2021)
- [Compositional Transformers for Scene Generation](https://mlsanity.com/paper/2111.08960/) (2021)
- [A Mixture-of-Expert Approach to RL-based Dialogue Management](https://mlsanity.com/paper/2206.00059/) (2022)
- [Measuring and Improving Semantic Diversity of Dialogue Generation](https://mlsanity.com/paper/2210.05725/) (2022)
- [Top-Down Semantic Refinement for Image Captioning](https://mlsanity.com/paper/2510.22391/) (2025)
- [Recent Advances in Neural Text Generation: A Task-Agnostic Survey](https://mlsanity.com/paper/2203.03047/) (2022)
- [Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks](https://mlsanity.com/paper/2210.05664/) (2022)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Maintaining Common Ground in Dynamic Environments](https://mlsanity.com/paper/2105.14207/) (2021)
- [Situated Dialogue Learning through Procedural Environment Generation](https://mlsanity.com/paper/2110.03262/) (2021)
- [TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate](https://mlsanity.com/paper/2605.13909/) (2026)
- [Towards a Progression-Aware Autonomous Dialogue Agent](https://mlsanity.com/paper/2205.03692/) (2022)
- [Target-constrained Bidirectional Planning for Generation of Target-oriented Proactive Dialogue](https://mlsanity.com/paper/2403.06063/) (2024)
- [Symbolic Planning and Code Generation for Grounded Dialogue](https://mlsanity.com/paper/2310.17140/) (2023)
- [We Argue to Agree: Towards Personality-Driven Argumentation-Based Negotiation Dialogue Systems for Tourism](https://mlsanity.com/paper/2509.11118/) (2025)
- [Targeted Data Acquisition for Evolving Negotiation Agents](https://mlsanity.com/paper/2106.07728/) (2021)
- [Towards a Neural Era in Dialogue Management for Collaboration: A Literature Survey](https://mlsanity.com/paper/2307.09021/) (2023)
- [WeaSuL: Weakly Supervised Dialogue Policy Learning: Reward Estimation for Multi-turn Dialogue](https://mlsanity.com/paper/2108.01487/) (2021)
- [Learning to Sell: Reinforcement Learning for Strategic Large Language Model Agents in Multi-Product Markets](https://mlsanity.com/paper/2609.33289/) (2026)

## Abstract (arXiv)

End-to-end models for goal-orientated dialogue are challenging to train, because linguistic and strategic aspects are entangled in latent state vectors. We introduce an approach to learning representations of messages in dialogues by maximizing the likelihood of subsequent sentences and actions, which decouples the semantics of the dialogue utterance from its linguistic realization. We then use these latent sentence representations for hierarchical language generation, planning and reinforcement learning. Experiments show that our approach increases the end-task reward achieved by the model, improves the effectiveness of long-term planning using rollouts, and allows self-play reinforcement learning to improve decision making without diverging from human language. Our hierarchical latent-variable model outperforms previous work both linguistically and strategically.