# Universal Language Model Fine-Tuning with Subword Tokenization for Polish

arXiv: https://arxiv.org/abs/1810.10222 · HTML: https://mlsanity.com/paper/1810.10222/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2018.

## Built on by (most cited first, 1 of 1)

- [Meta-aware Learning in text-to-SQL Large Language Model](https://mlsanity.com/paper/2505.18929/) (2025)

## Abstract (arXiv)

Universal Language Model for Fine-tuning [arXiv:1801.06146] (ULMFiT) is one of the first NLP methods for efficient inductive transfer learning. Unsupervised pretraining results in improvements on many NLP tasks for English. In this paper, we describe a new method that uses subword tokenization to adapt ULMFiT to languages with high inflection. Our approach results in a new state-of-the-art for the Polish language, taking first place in Task 3 of PolEval'18. After further training, our final model outperformed the second best model by 35%. We have open-sourced our pretrained models and code.