# Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation

arXiv: https://arxiv.org/abs/1607.00970 · HTML: https://mlsanity.com/paper/1607.00970/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2016.

## Built on by (most cited first, 30 of 35)

- [From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning](https://mlsanity.com/paper/2308.12032/) (2023)
- [Surface Form Competition: Why the Highest Probability Answer Isn't Always Right](https://mlsanity.com/paper/2104.08315/) (2021)
- [Benchmarking Complex Instruction-Following with Multiple Constraints Composition](https://mlsanity.com/paper/2407.03978/) (2024)
- [Parallel Refinements for Lexically Constrained Text Generation with BART](https://mlsanity.com/paper/2109.12487/) (2021)
- [Mixture of Soft Prompts for Controllable Data Generation](https://mlsanity.com/paper/2303.01580/) (2023)
- [Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction](https://mlsanity.com/paper/2106.03044/) (2021)
- [Why is constrained neural language generation particularly challenging?](https://mlsanity.com/paper/2206.05395/) (2022)
- [Thinking Clearly, Talking Fast: Concept-Guided Non-Autoregressive Generation for Open-Domain Dialogue Systems](https://mlsanity.com/paper/2109.04084/) (2021)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [Flexible-length Text Infilling for Discrete Diffusion Models](https://mlsanity.com/paper/2506.13579/) (2025)
- [An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation](https://mlsanity.com/paper/2209.14627/) (2022)
- [Show Me How To Revise: Improving Lexically Constrained Sentence Generation with XLNet](https://mlsanity.com/paper/2109.05797/) (2021)
- [An Empirical Study on the Overlapping Problem of Open-Domain Dialogue Datasets](https://mlsanity.com/paper/2201.06219/) (2022)
- [Contextualized Scene Imagination for Generative Commonsense Reasoning](https://mlsanity.com/paper/2112.06318/) (2021)
- [AutoTemplate: A Simple Recipe for Lexically Constrained Text Generation](https://mlsanity.com/paper/2211.08387/) (2022)
- [Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillation](https://mlsanity.com/paper/2102.10780/) (2021)
- [Controllable Dialogue Generation with Disentangled Multi-grained Style Specification and Attribute Consistency Reward](https://mlsanity.com/paper/2109.06717/) (2021)
- [Incorporating Causal Analysis into Diversified and Logical Response Generation](https://mlsanity.com/paper/2209.09482/) (2022)
- [Transformer-based Lexically Constrained Headline Generation](https://mlsanity.com/paper/2109.07080/) (2021)
- [FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning](https://mlsanity.com/paper/2109.10510/) (2021)
- [Simulated Annealing for Emotional Dialogue Systems](https://mlsanity.com/paper/2109.10715/) (2021)
- [Hierarchical Inductive Transfer for Continual Dialogue Learning](https://mlsanity.com/paper/2203.10484/) (2022)
- [SituatedGen: Incorporating Geographical and Temporal Contexts into Generative Commonsense Reasoning](https://mlsanity.com/paper/2306.12552/) (2023)
- [An Ion Exchange Mechanism Inspired Story Ending Generator for Different Characters](https://mlsanity.com/paper/2209.00200/) (2022)
- [Enhancing Pre-trained Models with Text Structure Knowledge for Question Generation](https://mlsanity.com/paper/2209.04179/) (2022)
- [An Overview on Controllable Text Generation via Variational Auto-Encoders](https://mlsanity.com/paper/2211.07954/) (2022)
- [On The Truthfulness of 'Surprisingly Likely' Responses of Large Language Models](https://mlsanity.com/paper/2311.07692/) (2023)
- [Effective Unsupervised Constrained Text Generation based on Perturbed Masking](https://mlsanity.com/paper/2404.15877/) (2024)
- [Evaluating Human-LLM Representation Alignment: A Case Study on Affective Sentence Generation for Augmentative and Alternative Communication](https://mlsanity.com/paper/2503.11881/) (2025)
- [HAMLET: A Hierarchical and Adaptive Multi-Agent Framework for Live Embodied Theatrics](https://mlsanity.com/paper/2507.15518/) (2025)

## Abstract (arXiv)

Using neural networks to generate replies in human-computer dialogue systems is attracting increasing attention over the past few years. However, the performance is not satisfactory: the neural network tends to generate safe, universally relevant replies which carry little meaning. In this paper, we propose a content-introducing approach to neural network-based generative dialogue systems. We first use pointwise mutual information (PMI) to predict a noun as a keyword, reflecting the main gist of the reply. We then propose seq2BF, a "sequence to backward and forward sequences" model, which generates a reply containing the given keyword. Experimental results show that our approach significantly outperforms traditional sequence-to-sequence models in terms of human evaluation and the entropy measure, and that the predicted keyword can appear at an appropriate position in the reply.