# Unsupervised Word Segmentation with Bi-directional Neural Language Model

arXiv: https://arxiv.org/abs/2103.01421 · HTML: https://mlsanity.com/paper/2103.01421/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 38 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Exploring the Limits of Language Modeling](https://mlsanity.com/paper/1602.02410/) (2016)
- [Sequence Modeling via Segmentations](https://mlsanity.com/paper/1702.07463/) (2017)
- [Segmental Recurrent Neural Networks](https://mlsanity.com/paper/1511.06018/) (2015)
- [Unsupervised Neural Word Segmentation for Chinese via Segmental Language Modeling](https://mlsanity.com/paper/1810.03167/) (2018)
- [Adversarial Multi-Criteria Learning for Chinese Word Segmentation](https://mlsanity.com/paper/1704.07556/) (2017)
- [Switch-LSTMs for Multi-Criteria Chinese Word Segmentation](https://mlsanity.com/paper/1812.08033/) (2018)
- [Effective Neural Solution for Multi-Criteria Word Segmentation](https://mlsanity.com/paper/1712.02856/) (2017)

## Built on by (most cited first, 5 of 5)

- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [A Masked Segmental Language Model for Unsupervised Natural Language Segmentation](https://mlsanity.com/paper/2104.07829/) (2021)
- [Subword Segmental Language Modelling for Nguni Languages](https://mlsanity.com/paper/2210.06525/) (2022)
- [Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages](https://mlsanity.com/paper/2110.08415/) (2021)
- [Segment First or Comprehend First? Explore the Limit of Unsupervised Word Segmentation with Large Language Models](https://mlsanity.com/paper/2505.19631/) (2025)

## Abstract (arXiv)

We present an unsupervised word segmentation model, in which the learning objective is to maximize the generation probability of a sentence given its all possible segmentation. Such generation probability can be factorized into the likelihood of each possible segment given the context in a recursive way. In order to better capture the long- and short-term dependencies, we propose to use bi-directional neural language models to better capture the features of segment's context. Two decoding algorithms are also described to combine the context features from both directions to generate the final segmentation, which helps to reconcile word boundary ambiguities. Experimental results showed that our context-sensitive unsupervised segmentation model achieved state-of-the-art at different evaluation settings on various data sets for Chinese, and the comparable result for Thai.