# A Network Perspective on Stratification of Multi-Label Data

arXiv: https://arxiv.org/abs/1704.08756 · HTML: https://mlsanity.com/paper/1704.08756/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2017.

## Built on by (most cited first, 30 of 37)

- [Tackling Data Bias in MUSIC-AVQA: Crafting a Balanced Dataset for Unbiased Question-Answering](https://mlsanity.com/paper/2310.06238/) (2023)
- [CAVES: A Dataset to facilitate Explainable Classification and Summarization of Concerns towards COVID Vaccines](https://mlsanity.com/paper/2204.13746/) (2022)
- [Calibration and generalizability of probabilistic models on low-data chemical datasets with DIONYSUS](https://mlsanity.com/paper/2212.01574/) (2022)
- [AnuraSet: A dataset for benchmarking Neotropical anuran calls identification in passive acoustic monitoring](https://mlsanity.com/paper/2307.06860/) (2023)
- [Navya3DSeg -- Navya 3D Semantic Segmentation Dataset & split generation for autonomous vehicles](https://mlsanity.com/paper/2302.08292/) (2023)
- [Stratified Sampling for Extreme Multi-Label Data](https://mlsanity.com/paper/2103.03494/) (2021)
- [Predicting Different Types of Subtle Toxicity in Unhealthy Online Conversations](https://mlsanity.com/paper/2106.03952/) (2021)
- [Multi-Modal Attribute Extraction for E-Commerce](https://mlsanity.com/paper/2203.03441/) (2022)
- [Generalizable, Fast, and Accurate DeepQSPR with fastprop](https://mlsanity.com/paper/2404.02058/) (2024)
- [SynthSOD: Developing an Heterogeneous Dataset for Orchestra Music Source Separation](https://mlsanity.com/paper/2409.10995/) (2024)
- [Latent Abstractions in Generative Diffusion Models](https://mlsanity.com/paper/2410.03368/) (2024)
- [EvoSplit: An evolutionary approach to split a multi-label data set into disjoint subsets](https://mlsanity.com/paper/2102.06154/) (2021)
- [Novel split quality measures for stratified multilabel Cross Validation with application to large and sparse gene ontology datasets](https://mlsanity.com/paper/2109.01425/) (2021)
- [An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialog](https://mlsanity.com/paper/2109.12212/) (2021)
- [Improving Transfer Learning with a Dual Image and Video Transformer for Multi-label Movie Trailer Genre Classification](https://mlsanity.com/paper/2210.07983/) (2022)
- [A Clinically-Grounded Two-Stage Framework for Renal CT Report Generation](https://mlsanity.com/paper/2506.23584/) (2025)
- [Clinical Language Understanding Evaluation (CLUE)](https://mlsanity.com/paper/2209.14377/) (2022)
- [Actionable Conversational Quality Indicators for Improving Task-Oriented Dialog Systems](https://mlsanity.com/paper/2109.11064/) (2021)
- [A Mixed-Domain Self-Attention Network for Multilabel Cardiac Irregularity Classification Using Reduced-Lead Electrocardiogram](https://mlsanity.com/paper/2204.13917/) (2022)
- [Hierarchical Classification of Research Fields in the "Web of Science" Using Deep Learning](https://mlsanity.com/paper/2302.00390/) (2023)
- [Fine-Tuning In-House Large Language Models to Infer Differential Diagnosis from Radiology Reports](https://mlsanity.com/paper/2410.09234/) (2024)
- [Stratify or Die: Rethinking Data Splits in Image Segmentation](https://mlsanity.com/paper/2509.21056/) (2025)
- [Multi-Label Quantification](https://mlsanity.com/paper/2211.08063/) (2022)
- [BarlowTwins-CXR : Enhancing Chest X-Ray abnormality localization in heterogeneous data with cross-domain self-supervised learning](https://mlsanity.com/paper/2402.06499/) (2024)
- [DeepNose: An Equivariant Convolutional Neural Network Predictive Of Human Olfactory Percepts](https://mlsanity.com/paper/2412.08747/) (2024)
- [Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Models](https://mlsanity.com/paper/2506.03321/) (2025)
- [A Language-Signal-Vision Multimodal Framework for Multitask Cardiac Analysis](https://mlsanity.com/paper/2508.13072/) (2025)
- [AI-Powered Detection of Inappropriate Language in Medical School Curricula](https://mlsanity.com/paper/2508.19883/) (2025)
- [Learning Sparse Label Couplings for Multilabel Chest X-Ray Diagnosis](https://mlsanity.com/paper/2511.07801/) (2025)
- [Not All Pretraining are Created Equal: Threshold Tuning and Class Weighting for Imbalanced Polarization Tasks in Low-Resource Settings](https://mlsanity.com/paper/2603.23534/) (2026)

## Abstract (arXiv)

In the recent years, we have witnessed the development of multi-label classification methods which utilize the structure of the label space in a divide and conquer approach to improve classification performance and allow large data sets to be classified efficiently. Yet most of the available data sets have been provided in train/test splits that did not account for maintaining a distribution of higher-order relationships between labels among splits or folds. We present a new approach to stratifying multi-label data for classification purposes based on the iterative stratification approach proposed by Sechidis et. al. in an ECML PKDD 2011 paper. Our method extends the iterative approach to take into account second-order relationships between labels. Obtained results are evaluated using statistical properties of obtained strata as presented by Sechidis. We also propose new statistical measures relevant to second-order quality: label pairs distribution, the percentage of label pairs without positive evidence in folds and label pair - fold pairs that have no positive evidence for the label pair. We verify the impact of new methods on classification performance of Binary Relevance, Label Powerset and a fast greedy community detection based label space partitioning classifier. Random Forests serve as base classifiers. We check the variation of the number of communities obtained per fold, and the stability of their modularity score. Second-Order Iterative Stratification is compared to standard k-fold, label set, and iterative stratification. The proposed approach lowers the variance of classification quality, improves label pair oriented measures and example distribution while maintaining a competitive quality in label-oriented measures. We also witness an increase in stability of network characteristics.