# Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation

arXiv: https://arxiv.org/abs/1508.02096 · HTML: https://mlsanity.com/paper/1508.02096/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2015.

## Built on by (most cited first, 28 of 28)

- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [RobeCzech: Czech RoBERTa, a monolingual contextualized language representation model](https://mlsanity.com/paper/2105.11314/) (2021)
- [Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models](https://mlsanity.com/paper/2203.10326/) (2022)
- [Dynamic Language Models for Continuously Evolving Content](https://mlsanity.com/paper/2106.06297/) (2021)
- [Conformal prediction for text infilling and part-of-speech prediction](https://mlsanity.com/paper/2111.02592/) (2021)
- [Capitalization and Punctuation Restoration: a Survey](https://mlsanity.com/paper/2111.10746/) (2021)
- [To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP](https://mlsanity.com/paper/2111.09618/) (2021)
- [The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?](https://mlsanity.com/paper/2601.07220/) (2026)
- [Recent Trends in Named Entity Recognition (NER)](https://mlsanity.com/paper/2101.11420/) (2021)
- [A Survey of Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2203.02982/) (2022)
- [Language Modelling via Learning to Rank](https://mlsanity.com/paper/2110.06961/) (2021)
- [What is the best recipe for character-level encoder-only modelling?](https://mlsanity.com/paper/2305.05461/) (2023)
- [Integrating Approaches to Word Representation](https://mlsanity.com/paper/2109.04876/) (2021)
- [Towards a Deep Multi-layered Dialectal Language Analysis: A Case Study of African-American English](https://mlsanity.com/paper/2206.08978/) (2022)
- [Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields](https://mlsanity.com/paper/2404.09383/) (2024)
- [Evaluating Models of Robust Word Recognition with Serial Reproduction](https://mlsanity.com/paper/2101.09788/) (2021)
- [Locally-Contextual Nonlinear CRFs for Sequence Labeling](https://mlsanity.com/paper/2103.16210/) (2021)
- [A Survey On Neural Word Embeddings](https://mlsanity.com/paper/2110.01804/) (2021)
- [Data Augmentation for Neural NLP](https://mlsanity.com/paper/2302.11412/) (2023)
- [UnibucKernel: Geolocating Swiss German Jodels Using Ensemble Learning](https://mlsanity.com/paper/2102.09379/) (2021)
- [UPB at SemEval-2021 Task 5: Virtual Adversarial Training for Toxic Spans Detection](https://mlsanity.com/paper/2104.08635/) (2021)
- [The boundaries of meaning: a case study in neural machine translation](https://mlsanity.com/paper/2210.00613/) (2022)
- [Leveraging knowledge graphs to update scientific word embeddings using latent semantic imputation](https://mlsanity.com/paper/2210.15358/) (2022)
- [Language Cognition and Language Computation -- Human and Machine Language Understanding](https://mlsanity.com/paper/2301.04788/) (2023)
- [Learning grounded word meaning representations on similarity graphs](https://mlsanity.com/paper/2109.03084/) (2021)
- [Part-of-Speech Tagging of Odia Language Using statistical and Deep Learning-Based Approaches](https://mlsanity.com/paper/2207.03256/) (2022)
- [Semantic Tagging with LSTM-CRF](https://mlsanity.com/paper/2301.12206/) (2023)
- [GWPT: A Green Word-Embedding-based POS Tagger](https://mlsanity.com/paper/2401.07475/) (2024)

## Abstract (arXiv)

We introduce a model for constructing vector representations of words by composing characters using bidirectional LSTMs. Relative to traditional word representation models that have independent vectors for each word type, our model requires only a single vector per character type and a fixed set of parameters for the compositional model. Despite the compactness of this model and, more importantly, the arbitrary nature of the form-function relationship in language, our "composed" word representations yield state-of-the-art results in language modeling and part-of-speech tagging. Benefits over traditional baselines are particularly pronounced in morphologically rich languages (e.g., Turkish).