# Synthetic Feature Augmentation Improves Generalization Performance of Language Models

arXiv: https://arxiv.org/abs/2501.06434 · HTML: https://mlsanity.com/paper/2501.06434/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 16 references that are in the map.
- First posted on arXiv 11 Jan 2025.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Data augmentation method for modeling health records with applications to clopidogrel treatment failure detection](https://mlsanity.com/paper/2402.18046/) (2024)

## Built on by (most cited first, 1 of 1)

- [CFMD: Dynamic Cross-layer Feature Fusion for Salient Object Detection](https://mlsanity.com/paper/2504.01326/) (2025)

## Abstract (arXiv)

Training and fine-tuning deep learning models, especially large language models (LLMs), on limited and imbalanced datasets poses substantial challenges. These issues often result in poor generalization, where models overfit to dominant classes and underperform on minority classes, leading to biased predictions and reduced robustness in real-world applications. To overcome these challenges, we propose augmenting features in the embedding space by generating synthetic samples using a range of techniques. By upsampling underrepresented classes, this method improves model performance and alleviates data imbalance. We validate the effectiveness of this approach across multiple open-source text classification benchmarks, demonstrating its potential to enhance model robustness and generalization in imbalanced data scenarios.