# From Characters to Words to in Between: Do We Capture Morphology?

arXiv: https://arxiv.org/abs/1704.08352 · HTML: https://mlsanity.com/paper/1704.08352/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2017.

## Built on by (most cited first, 13 of 13)

- [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://mlsanity.com/paper/2103.06874/) (2021)
- [Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold](https://mlsanity.com/paper/2206.09755/) (2022)
- [BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages](https://mlsanity.com/paper/2203.08954/) (2022)
- [Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation](https://mlsanity.com/paper/2210.14389/) (2022)
- [How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology?](https://mlsanity.com/paper/2109.01100/) (2021)
- [Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction](https://mlsanity.com/paper/2306.06804/) (2023)
- [What is the best recipe for character-level encoder-only modelling?](https://mlsanity.com/paper/2305.05461/) (2023)
- [Tokenization with Factorized Subword Encoding](https://mlsanity.com/paper/2306.07764/) (2023)
- [Responsible Multilingual Large Language Models: A Survey of Development, Applications, and Societal Impact](https://mlsanity.com/paper/2410.17532/) (2024)
- [AdaptBPE: From General Purpose to Specialized Tokenizers](https://mlsanity.com/paper/2601.21665/) (2026)
- [The boundaries of meaning: a case study in neural machine translation](https://mlsanity.com/paper/2210.00613/) (2022)
- [HDND: Hierarchical Dynamic Neural Decoding for Multilingual Word/Character Retrieval from Non-Invasive Brain Recordings](https://mlsanity.com/paper/2609.24095/) (2026)
- [BERT Cannot Align Characters](https://mlsanity.com/paper/2109.09700/) (2021)

## Abstract (arXiv)

Words can be represented by composing the representations of subword units such as word segments, characters, and/or character n-grams. While such representations are effective and may capture the morphological regularities of words, they have not been systematically compared, and it is not understood how they interact with different morphological typologies. On a language modeling task, we present experiments that systematically vary (1) the basic unit of representation, (2) the composition of these representations, and (3) the morphological typology of the language modeled. Our results extend previous findings that character representations are effective across typologies, and we find that a previously unstudied combination of character trigram representations composed with bi-LSTMs outperforms most others. But we also find room for improvement: none of the character-level models match the predictive accuracy of a model with access to true morphological analyses, even when learned from an order of magnitude more data.