# inversedMixup: Data Augmentation via Inverting Mixed Embeddings

arXiv: https://arxiv.org/abs/2601.21543 · HTML: https://mlsanity.com/paper/2601.21543/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias](https://mlsanity.com/paper/2306.15895/) (2023)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Augmenting Data with Mixup for Sentence Classification: An Empirical Study](https://mlsanity.com/paper/1905.08941/) (2019)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://mlsanity.com/paper/2310.07849/) (2023)
- [Dual Mixup Regularized Learning for Adversarial Domain Adaptation](https://mlsanity.com/paper/2007.03141/) (2020)
- [Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence](https://mlsanity.com/paper/2305.03010/) (2023)
- [Language Model Inversion](https://mlsanity.com/paper/2311.13647/) (2023)
- [MixCo: Mix-up Contrastive Learning for Visual Representation](https://mlsanity.com/paper/2010.06300/) (2020)
- [Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks](https://mlsanity.com/paper/1909.11515/) (2019)
- [Text Embedding Inversion Security for Multilingual Language Models](https://mlsanity.com/paper/2401.12192/) (2024)
- [Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Tasks](https://mlsanity.com/paper/2202.13840/) (2022)
- [Ada-Instruct: Adapting Instruction Generators for Complex Reasoning](https://mlsanity.com/paper/2310.04484/) (2023)
- [Understanding and Mitigating the Threat of Vec2Text to Dense Retrieval Systems](https://mlsanity.com/paper/2402.12784/) (2024)
- [Simple-Sampling and Hard-Mixup with Prototypes to Rebalance Contrastive Learning for Text Classification](https://mlsanity.com/paper/2405.11524/) (2024)
- [Adversarial Word Dilution as Text Data Augmentation in Low-Resource Regime](https://mlsanity.com/paper/2305.09287/) (2023)

## Abstract (arXiv)

Mixup generates augmented samples by linearly interpolating inputs and labels with a controllable ratio. However, since it operates at the latent embedding level, the resulting samples are not human-interpretable. In contrast, LLM-based augmentation methods produce sentences via prompts at the token level, yielding readable outputs but offering limited control over the generation process. Inspired by recent advances in LLM inversion, which reconstructs natural language from embeddings and helps bridge the gap between latent embedding space and discrete token space, we propose inversedMixup, a unified framework that combines the controllability of Mixup with the interpretability of LLM-based generation. Specifically, inversedMixup aligns the output embedding space of a task-specific model with the input embedding space of an LLM, so that mixed embeddings can be reconstructed, under a controllable mixing ratio, into human-interpretable sentences. This interpretability provides the first empirical evidence of the manifold intrusion phenomenon in text Mixup. Building on this, we extend inversedMixup into a three-stage data augmentation method, and introduce a simple yet effective strategy to mitigate manifold intrusion during augmentation. Extensive experiments demonstrate the effectiveness and generalizability of our approach in both few-shot and fully supervised scenarios.