# UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generation

arXiv: https://arxiv.org/abs/2405.01022 · HTML: https://mlsanity.com/paper/2405.01022/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 2 May 2024.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Textbooks Are All You Need II: phi-1.5 technical report](https://mlsanity.com/paper/2309.05463/) (2023)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://mlsanity.com/paper/2202.07922/) (2022)
- [SemEval-2017 Task 4: Sentiment Analysis in Twitter](https://mlsanity.com/paper/1912.00741/) (2019)
- [Generating Training Data with Language Models: Towards Zero-Shot Language Understanding](https://mlsanity.com/paper/2202.04538/) (2022)
- [SelfReg: Self-supervised Contrastive Regularization for Domain Generalization](https://mlsanity.com/paper/2104.09841/) (2021)
- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)
- [Not Enough Data? Deep Learning to the Rescue!](https://mlsanity.com/paper/1911.03118/) (2019)
- [Learning to Diversify for Single Domain Generalization](https://mlsanity.com/paper/2108.11726/) (2021)
- [ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback](https://mlsanity.com/paper/2210.12329/) (2022)
- [Multinomial Adversarial Networks for Multi-Domain Text Classification](https://mlsanity.com/paper/1802.05694/) (2018)
- [Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits](https://mlsanity.com/paper/2001.04362/) (2020)
- [Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance](https://mlsanity.com/paper/2311.01108/) (2023)
- [Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification](https://mlsanity.com/paper/2109.00523/) (2021)
- [Test-Time Self-Adaptive Small Language Models for Question Answering](https://mlsanity.com/paper/2310.13307/) (2023)
- [Handling Realistic Label Noise in BERT Text Classification](https://mlsanity.com/paper/2305.16337/) (2023)

## Built on by (most cited first, 3 of 3)

- [Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text Classification](https://mlsanity.com/paper/2410.21526/) (2024)
- [Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation](https://mlsanity.com/paper/2510.08962/) (2025)
- [Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models](https://mlsanity.com/paper/2506.07424/) (2025)

## Abstract (arXiv)

Although pre-trained language models have exhibited great flexibility and versatility with prompt-based few-shot learning, they suffer from the extensive parameter size and limited applicability for inference. Recent studies have suggested that PLMs be used as dataset generators and a tiny task-specific model be trained to achieve efficient inference. However, their applicability to various domains is limited because they tend to generate domain-specific datasets. In this work, we propose a novel approach to universal domain generalization that generates a dataset regardless of the target domain. This allows for generalization of the tiny task model to any domain that shares the label space, thus enhancing the real-world applicability of the dataset generation paradigm. Our experiments indicate that the proposed method accomplishes generalizability across various domains while using a parameter set that is orders of magnitude smaller than PLMs.