# Urdu Word Segmentation using Conditional Random Fields (CRFs)

arXiv: https://arxiv.org/abs/1806.05432 · HTML: https://mlsanity.com/paper/1806.05432/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2018.

## Built on by (most cited first, 4 of 4)

- [Exploiting Transliterated Words for Finding Similarity in Inter-Language News Articles using Machine Learning](https://mlsanity.com/paper/2206.11860/) (2022)
- [Riazi-8B: An Urdu Large Language Model for Mathematical Reasoning](https://mlsanity.com/paper/2606.25568/) (2026)
- [A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages](https://mlsanity.com/paper/2609.18739/) (2026)
- [Multi-Task Learning by using Contextualized Word Representations for Syntactic Parsing of a Morphologically Rich Language](https://mlsanity.com/paper/2609.29855/) (2026)

## Abstract (arXiv)

State-of-the-art Natural Language Processing algorithms rely heavily on efficient word segmentation. Urdu is amongst languages for which word segmentation is a complex task as it exhibits space omission as well as space insertion issues. This is partly due to the Arabic script which although cursive in nature, consists of characters that have inherent joining and non-joining attributes regardless of word boundary. This paper presents a word segmentation system for Urdu which uses a Conditional Random Field sequence modeler with orthographic, linguistic and morphological features. Our proposed model automatically learns to predict white space as word boundary as well as Zero Width Non-Joiner (ZWNJ) as sub-word boundary. Using a manually annotated corpus, our model achieves F1 score of 0.97 for word boundary identification and 0.85 for sub-word boundary identification tasks. We have made our code and corpus publicly available to make our results reproducible.