# Sequence Modeling via Segmentations

arXiv: https://arxiv.org/abs/1702.07463 · HTML: https://mlsanity.com/paper/1702.07463/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2017.

## Built on by (most cited first, 18 of 18)

- [Charformer: Fast Character Transformers via Gradient-based Subword Tokenization](https://mlsanity.com/paper/2106.12672/) (2021)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning](https://mlsanity.com/paper/2202.13675/) (2022)
- [Structured Reordering for Modeling Latent Alignments in Sequence Transduction](https://mlsanity.com/paper/2106.03257/) (2021)
- [A Masked Segmental Language Model for Unsupervised Natural Language Segmentation](https://mlsanity.com/paper/2104.07829/) (2021)
- [Subword Segmental Language Modelling for Nguni Languages](https://mlsanity.com/paper/2210.06525/) (2022)
- [CompILE: Compositional Imitation Learning and Execution](https://mlsanity.com/paper/1812.01483/) (2018)
- [Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation](https://mlsanity.com/paper/2305.07005/) (2023)
- [Unsupervised Word Segmentation with Bi-directional Neural Language Model](https://mlsanity.com/paper/2103.01421/) (2021)
- [Triples-to-isiXhosa (T2X): Addressing the Challenges of Low-Resource Agglutinative Data-to-Text Generation](https://mlsanity.com/paper/2403.07567/) (2024)
- [Learning to Express in Knowledge-Grounded Conversation](https://mlsanity.com/paper/2204.05805/) (2022)
- [Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages](https://mlsanity.com/paper/2110.08415/) (2021)
- [SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation](https://mlsanity.com/paper/2307.16400/) (2023)
- [Segment First or Comprehend First? Explore the Limit of Unsupervised Word Segmentation with Large Language Models](https://mlsanity.com/paper/2505.19631/) (2025)
- [H-Net++: Hierarchical Dynamic Chunking for Tokenizer-Free Language Modelling in Morphologically-Rich Languages](https://mlsanity.com/paper/2508.05628/) (2025)
- [Right Reset: Chunking by Prefix Removal](https://mlsanity.com/paper/2608.04330/) (2026)
- [Sentence Splitter: Uncovering Latent Factual Structure for Self-Supervised Learning](https://mlsanity.com/paper/2607.19845/) (2026)
- [Subword Segmental BabyLMs: Learning to Tokenise for Sample-Efficient Pretraining](https://mlsanity.com/paper/2609.01151/) (2026)

## Abstract (arXiv)

Segmental structure is a common pattern in many types of sequences such as phrases in human languages. In this paper, we present a probabilistic model for sequences via their segmentations. The probability of a segmented sequence is calculated as the product of the probabilities of all its segments, where each segment is modeled using existing tools such as recurrent neural networks. Since the segmentation of a sequence is usually unknown in advance, we sum over all valid segmentations to obtain the final probability for the sequence. An efficient dynamic programming algorithm is developed for forward and backward computations without resorting to any approximation. We demonstrate our approach on text segmentation and speech recognition tasks. In addition to quantitative results, we also show that our approach can discover meaningful segments in their respective application contexts.