# The Effect of Data Partitioning Strategy on Model Generalizability: A Case Study of Morphological Segmentation

arXiv: https://arxiv.org/abs/2404.09371 · HTML: https://mlsanity.com/paper/2404.09371/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 42 references that are in the map.
- First posted on arXiv 14 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [We Need to Talk About Random Splits](https://mlsanity.com/paper/2005.00636/) (2020)
- [BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages](https://mlsanity.com/paper/2203.08954/) (2022)
- [Fortification of Neural Morphological Segmentation Models for Polysynthetic Minimal-Resource Languages](https://mlsanity.com/paper/1804.06024/) (2018)
- [Tackling the Low-resource Challenge for Canonical Segmentation](https://mlsanity.com/paper/2010.02804/) (2020)
- [Investigating data partitioning strategies for crosslinguistic low-resource ASR evaluation](https://mlsanity.com/paper/2208.12888/) (2022)
- [Labeled Morphological Segmentation with Semi-Markov Models](https://mlsanity.com/paper/2404.08997/) (2024)

## Built on by (most cited first, 1 of 1)

- [Beyond statistical significance: Quantifying uncertainty and statistical variability in multilingual and multitask NLP evaluation](https://mlsanity.com/paper/2509.22612/) (2025)

## Abstract (arXiv)

Recent work to enhance data partitioning strategies for more realistic model evaluation face challenges in providing a clear optimal choice. This study addresses these challenges, focusing on morphological segmentation and synthesizing limitations related to language diversity, adoption of multiple datasets and splits, and detailed model comparisons. Our study leverages data from 19 languages, including ten indigenous or endangered languages across 10 language families with diverse morphological systems (polysynthetic, fusional, and agglutinative) and different degrees of data availability. We conduct large-scale experimentation with varying sized combinations of training and evaluation sets as well as new test data. Our results show that, when faced with new test data: (1) models trained from random splits are able to achieve higher numerical scores; (2) model rankings derived from random splits tend to generalize more consistently.