# DAIL: Data Augmentation for In-Context Learning via Self-Paraphrase

arXiv: https://arxiv.org/abs/2311.03319 · HTML: https://mlsanity.com/paper/2311.03319/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 6 Nov 2023.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [Selective Annotation Makes Language Models Better Few-Shot Learners](https://mlsanity.com/paper/2209.01975/) (2022)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Human-like Summarization Evaluation with ChatGPT](https://mlsanity.com/paper/2304.02554/) (2023)
- [On the Inference Calibration of Neural Machine Translation](https://mlsanity.com/paper/2005.00963/) (2020)
- [LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification](https://mlsanity.com/paper/2205.12528/) (2022)
- [tasksource: A Dataset Harmonization Framework for Streamlined NLP Multi-Task Learning and Evaluation](https://mlsanity.com/paper/2301.05948/) (2023)
- [SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank](https://mlsanity.com/paper/2305.14696/) (2023)

## Built on by (most cited first, 6 of 6)

- [Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning](https://mlsanity.com/paper/2403.20046/) (2024)
- [DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literature](https://mlsanity.com/paper/2405.04819/) (2024)
- [Contextualization Distillation from Large Language Model for Knowledge Graph Completion](https://mlsanity.com/paper/2402.01729/) (2024)
- [LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanations](https://mlsanity.com/paper/2401.12576/) (2024)
- [READ: Improving Relation Extraction from an ADversarial Perspective](https://mlsanity.com/paper/2404.02931/) (2024)
- [Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings](https://mlsanity.com/paper/2503.05620/) (2025)

## Abstract (arXiv)

In-Context Learning (ICL) combined with pre-trained large language models has achieved promising results on various NLP tasks. However, ICL requires high-quality annotated demonstrations which might not be available in real-world scenarios. To overcome this limitation, we propose Data Augmentation for In-Context Learning (DAIL). DAIL leverages the intuition that large language models are more familiar with the content generated by themselves. It first utilizes the language model to generate paraphrases of the test sample and employs majority voting to determine the final result based on individual predictions. Our extensive empirical evaluation shows that DAIL outperforms the standard ICL method and other ensemble-based methods in the low-resource scenario. Additionally, we explore the use of voting consistency as a confidence score of the model when the logits of predictions are inaccessible. We believe our work will stimulate further research on ICL in low-resource settings.