# Unwritten Languages Demand Attention Too! Word Discovery with Encoder-Decoder Models

arXiv: https://arxiv.org/abs/1709.05631 · HTML: https://mlsanity.com/paper/1709.05631/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2017.

## Built on by (most cited first, 3 of 3)

- [On the Difficulty of Segmenting Words with Attention](https://mlsanity.com/paper/2109.10107/) (2021)
- [AlloST: Low-resource Speech Translation without Source Transcription](https://mlsanity.com/paper/2105.00171/) (2021)
- [Unsupervised Word Segmentation from Discrete Speech Units in Low-Resource Settings](https://mlsanity.com/paper/2106.04298/) (2021)

## Abstract (arXiv)

Word discovery is the task of extracting words from unsegmented text. In this paper we examine to what extent neural networks can be applied to this task in a realistic unwritten language scenario, where only small corpora and limited annotations are available. We investigate two scenarios: one with no supervision and another with limited supervision with access to the most frequent words. Obtained results show that it is possible to retrieve at least 27% of the gold standard vocabulary by training an encoder-decoder neural machine translation system with only 5,157 sentences. This result is close to those obtained with a task-specific Bayesian nonparametric model. Moreover, our approach has the advantage of generating translation alignments, which could be used to create a bilingual lexicon. As a future perspective, this approach is also well suited to work directly from speech.