# Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup

arXiv: https://arxiv.org/abs/2112.06204 · HTML: https://mlsanity.com/paper/2112.06204/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 12 Dec 2021.

## Builds on (most cited first, 24 of 24)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Explain Yourself! Leveraging Language Models for Commonsense Reasoning](https://mlsanity.com/paper/1906.02361/) (2019)
- [Textual Explanations for Self-Driving Vehicles](https://mlsanity.com/paper/1807.11546/) (2018)
- [Multimodal Explanations: Justifying Decisions and Pointing to the Evidence](https://mlsanity.com/paper/1802.08129/) (2018)
- [Measuring Association Between Labels and Free-Text Rationales](https://mlsanity.com/paper/2010.12762/) (2020)
- [Generating Visual Explanations](https://mlsanity.com/paper/1603.08507/) (2016)
- [WT5?! Training Text-to-Text Models to Explain their Predictions](https://mlsanity.com/paper/2004.14546/) (2020)
- [NILE : Natural Language Inference with Faithful Natural Language Explanations](https://mlsanity.com/paper/2005.12116/) (2020)
- [Few-shot Natural Language Generation for Task-Oriented Dialog](https://mlsanity.com/paper/2002.12328/) (2020)
- [e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks](https://mlsanity.com/paper/2105.03761/) (2021)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations](https://mlsanity.com/paper/1910.03065/) (2019)
- [Towards Explainable NLP: A Generative Explanation Framework for Text Classification](https://mlsanity.com/paper/1811.00196/) (2018)
- [Few-Shot NLG with Pre-Trained Language Model](https://mlsanity.com/paper/1904.09521/) (2019)
- [WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge](https://mlsanity.com/paper/2005.05763/) (2020)
- [SemEval-2020 Task 4: Commonsense Validation and Explanation](https://mlsanity.com/paper/2007.00236/) (2020)
- [Explaining Question Answering Models through Text Generation](https://mlsanity.com/paper/2004.05569/) (2020)
- [Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations](https://mlsanity.com/paper/2106.13876/) (2021)
- [Few-Shot Dialogue Generation Without Annotated Data: A Transfer Learning Approach](https://mlsanity.com/paper/1908.05854/) (2019)
- [Case-Based Abductive Natural Language Inference](https://mlsanity.com/paper/2009.14539/) (2020)

## Built on by (most cited first, 16 of 16)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Reframing Human-AI Collaboration for Generating Free-Text Explanations](https://mlsanity.com/paper/2112.08674/) (2021)
- [SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales](https://mlsanity.com/paper/2405.20974/) (2024)
- [FLUTE: Figurative Language Understanding through Textual Explanations](https://mlsanity.com/paper/2205.12404/) (2022)
- [The Defeat of the Winograd Schema Challenge](https://mlsanity.com/paper/2201.02387/) (2022)
- [Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda](https://mlsanity.com/paper/2404.09554/) (2024)
- [ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?](https://mlsanity.com/paper/2403.17368/) (2024)
- [S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learning](https://mlsanity.com/paper/2309.02155/) (2023)
- [PromptExp: Multi-granularity Prompt Explanation of Large Language Models](https://mlsanity.com/paper/2410.13073/) (2024)
- [Understanding and Predicting Human Label Variation in Natural Language Inference through Explanation](https://mlsanity.com/paper/2304.12443/) (2023)
- [Take It Easy: Label-Adaptive Self-Rationalization for Fact Verification and Explanation Generation](https://mlsanity.com/paper/2410.04002/) (2024)
- [Towards Transparent AI: A Survey on Explainable Language Models](https://mlsanity.com/paper/2509.21631/) (2025)
- [SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations](https://mlsanity.com/paper/2305.13235/) (2023)
- [LEKA:LLM-Enhanced Knowledge Augmentation](https://mlsanity.com/paper/2501.17802/) (2025)
- [Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related tasks](https://mlsanity.com/paper/2502.04797/) (2025)

## Abstract (arXiv)

Training a model to provide natural language explanations (NLEs) for its predictions usually requires the acquisition of task-specific NLEs, which is time- and resource-consuming. A potential solution is the few-shot out-of-domain transfer of NLEs from a parent task with many NLEs to a child task. In this work, we examine the setup in which the child task has few NLEs but abundant labels. We establish four few-shot transfer learning methods that cover the possible fine-tuning combinations of the labels and NLEs for the parent and child tasks. We transfer explainability from a large natural language inference dataset (e-SNLI) separately to two child tasks: (1) hard cases of pronoun resolution, where we introduce the small-e-WinoGrande dataset of NLEs on top of the WinoGrande dataset, and (2)~commonsense validation (ComVE). Our results demonstrate that the parent task helps with NLE generation and we establish the best methods for this setup.