# Attending Form and Context to Generate Specialized Out-of-VocabularyWords Representations

arXiv: https://arxiv.org/abs/1912.06876 · HTML: https://mlsanity.com/paper/1912.06876/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2019.

## Built on by (most cited first, 1 of 1)

- [Integrating Approaches to Word Representation](https://mlsanity.com/paper/2109.04876/) (2021)

## Abstract (arXiv)

We propose a new contextual-compositional neural network layer that handles out-of-vocabulary (OOV) words in natural language processing (NLP) tagging tasks. This layer consists of a model that attends to both the character sequence and the context in which the OOV words appear. We show that our model learns to generate task-specific textitand sentence-dependent OOV word representations without the need for pre-training on an embedding table, unlike previous attempts. We insert our layer in the state-of-the-art tagging model of citetplank2016multilingual and thoroughly evaluate its contribution on 23 different languages on the task of jointly tagging part-of-speech and morphosyntactic attributes. Our OOV handling method successfully improves performances of this model on every language but one to achieve a new state-of-the-art on the Universal Dependencies Dataset 1.4.