# Learning to Start for Sequence to Sequence Architecture

arXiv: https://arxiv.org/abs/1608.05554 · HTML: https://mlsanity.com/paper/1608.05554/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2016.

## Built on by (most cited first, 1 of 1)

- [A Static and Dynamic Attention Framework for Multi Turn Dialogue Generation](https://mlsanity.com/paper/2410.20766/) (2024)

## Abstract (arXiv)

The sequence to sequence architecture is widely used in the response generation and neural machine translation to model the potential relationship between two sentences. It typically consists of two parts: an encoder that reads from the source sentence and a decoder that generates the target sentence word by word according to the encoder's output and the last generated word. However, it faces to the cold start problem when generating the first word as there is no previous word to refer. Existing work mainly use a special start symbol </s>to generate the first word. An obvious drawback of these work is that there is not a learnable relationship between words and the start symbol. Furthermore, it may lead to the error accumulation for decoding when the first word is incorrectly generated. In this paper, we proposed a novel approach to learning to generate the first word in the sequence to sequence architecture rather than using the start symbol. Experimental results on the task of response generation of short text conversation show that the proposed approach outperforms the state-of-the-art approach in both of the automatic and manual evaluations.