# SegMix: A Simple Structure-Aware Data Augmentation Method

arXiv: https://arxiv.org/abs/2311.09505 · HTML: https://mlsanity.com/paper/2311.09505/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [MasakhaNER: Named Entity Recognition for African Languages](https://mlsanity.com/paper/2103.11811/) (2021)
- [Extending Multilingual BERT to Low-Resource Languages](https://mlsanity.com/paper/2004.13640/) (2020)
- [Edinburgh Neural Machine Translation Systems for WMT 16](https://mlsanity.com/paper/1606.02891/) (2016)
- [An Analysis of Simple Data Augmentation for Named Entity Recognition](https://mlsanity.com/paper/2010.11683/) (2020)
- [SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation](https://mlsanity.com/paper/1808.07512/) (2018)
- [Sequence-Level Mixed Sample Data Augmentation](https://mlsanity.com/paper/2011.09039/) (2020)
- [AdvAug: Robust Adversarial Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2006.11834/) (2020)
- [GenIE: Generative Information Extraction](https://mlsanity.com/paper/2112.08340/) (2021)
- [Data Augmentation via Dependency Tree Morphing for Low-Resource Languages](https://mlsanity.com/paper/1903.09460/) (2019)
- [Local Additivity Based Data Augmentation for Semi-supervised NER](https://mlsanity.com/paper/2010.01677/) (2020)
- [Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation](https://mlsanity.com/paper/1601.03651/) (2016)
- [Snippext: Semi-supervised Opinion Mining with Augmented Data](https://mlsanity.com/paper/2002.03049/) (2020)
- [Substructure Substitution: Structured Data Augmentation for NLP](https://mlsanity.com/paper/2101.00411/) (2021)
- [Soft Gazetteers for Low-Resource Named Entity Recognition](https://mlsanity.com/paper/2005.01866/) (2020)
- [Improving BERT Model Using Contrastive Learning for Biomedical Relation Extraction](https://mlsanity.com/paper/2104.13913/) (2021)

## Abstract (arXiv)

Interpolation-based Data Augmentation (DA) methods (Mixup) linearly interpolate the inputs and labels of two or more training examples. Mixup has more recently been adapted to the field of Natural Language Processing (NLP), mainly for sequence labeling tasks. However, such a simple adoption yields mixed or unstable improvements over the baseline models. We argue that the direct-adoption methods do not account for structures in NLP tasks. To this end, we propose SegMix, a collection of interpolation-based DA algorithms that can adapt to task-specific structures. SegMix poses fewer constraints on data structures, is robust to various hyperparameter settings, applies to more task settings, and adds little computational overhead. In the algorithm's core, we apply interpolation methods on task-specific meaningful segments, in contrast to applying them on sequences as in prior work. We find SegMix to be a flexible framework that combines rule-based DA methods with interpolation-based methods, creating interesting mixtures of DA techniques. We show that SegMix consistently improves performance over strong baseline models in Named Entity Recognition (NER) and Relation Extraction (RE) tasks, especially under data-scarce settings. Furthermore, this method is easy to implement and adds negligible training overhead.