# Reusing Weights in Subword-aware Neural Language Models

arXiv: https://arxiv.org/abs/1802.08375 · HTML: https://mlsanity.com/paper/1802.08375/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2018.

## Built on by (most cited first, 1 of 1)

- [Gradient Descent Fails to Learn High-frequency Functions and Modular Arithmetic](https://mlsanity.com/paper/2310.12660/) (2023)

## Abstract (arXiv)

We propose several ways of reusing subword embeddings and other weights in subword-aware neural language models. The proposed techniques do not benefit a competitive character-aware model, but some of them improve the performance of syllable- and morpheme-aware models while showing significant reductions in model sizes. We discover a simple hands-on principle: in a multi-layer input embedding model, layers should be tied consecutively bottom-up if reused at output. Our best morpheme-aware model with properly reused weights beats the competitive word-level model by a large margin across multiple languages and has 20%-87% fewer parameters.