# Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data

arXiv: https://arxiv.org/abs/2212.09171 · HTML: https://mlsanity.com/paper/2212.09171/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 108 references that are in the map.
- First posted on arXiv 18 Dec 2022.

## Builds on (most cited first, 30 of 61)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [Why Normalizing Flows Fail to Detect Out-of-Distribution Data](https://mlsanity.com/paper/2006.08545/) (2020)
- [A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection](https://mlsanity.com/paper/2106.09022/) (2021)
- [MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines](https://mlsanity.com/paper/2007.12720/) (2020)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019)
- [The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT](https://mlsanity.com/paper/2010.06354/) (2020)
- [Types of Out-of-Distribution Texts and How to Detect Them](https://mlsanity.com/paper/2109.06827/) (2021)

## Built on by (most cited first, 21 of 21)

- [Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph](https://mlsanity.com/paper/2406.15627/) (2024)
- [SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain](https://mlsanity.com/paper/2407.19584/) (2024)
- [Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home](https://mlsanity.com/paper/2501.12835/) (2025)
- [TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning](https://mlsanity.com/paper/2505.11737/) (2025)
- [Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models](https://mlsanity.com/paper/2502.14427/) (2025)
- [Unsupervised Layer-wise Score Aggregation for Textual OOD Detection](https://mlsanity.com/paper/2302.09852/) (2023)
- [Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks](https://mlsanity.com/paper/2305.10284/) (2023)
- [SAUP: Situation Awareness Uncertainty Propagation on LLM Agent](https://mlsanity.com/paper/2412.01033/) (2024)
- [Enhanced Hallucination Detection in Neural Machine Translation through Simple Detector Aggregation](https://mlsanity.com/paper/2402.13331/) (2024)
- [Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models](https://mlsanity.com/paper/2310.13998/) (2023)
- [WeiPer: OOD Detection using Weight Perturbations of Class Projections](https://mlsanity.com/paper/2405.17164/) (2024)
- [What If the Input is Expanded in OOD Detection?](https://mlsanity.com/paper/2410.18472/) (2024)
- [Toward Stronger Textual Attack Detectors](https://mlsanity.com/paper/2310.14001/) (2023)
- [Semantic Calibration Prevails Where Token Confidence Fails: Benchmarking Long-Form Scientific QA](https://mlsanity.com/paper/2602.00279/) (2026)
- [Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs](https://mlsanity.com/paper/2505.23996/) (2025)
- [Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism](https://mlsanity.com/paper/2402.12997/) (2024)
- [ALIEN: Aligned Entropy Head for Improving Uncertainty Estimation of LLMs](https://mlsanity.com/paper/2505.15443/) (2025)
- [Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation](https://mlsanity.com/paper/2601.15645/) (2026)
- [How Do Uncertainty Estimators Align with Hallucination Occurrence in LLMs?](https://mlsanity.com/paper/2605.27016/) (2026)
- [GUT: Quantifying and Optimizing the Reasoning Uncertainty of LLMs via Graph Complexity](https://mlsanity.com/paper/2609.05284/) (2026)
- [Characterizing Overconfident Failure in LLM-Based Code Generation](https://mlsanity.com/paper/2610.11300/) (2026)

## Abstract (arXiv)

Implementing effective control mechanisms to ensure the proper functioning and security of deployed NLP models, from translation to chatbots, is essential. A key ingredient to ensure safe system behaviour is Out-Of-Distribution (OOD) detection, which aims to detect whether an input sample is statistically far from the training distribution. Although OOD detection is a widely covered topic in classification tasks, most methods rely on hidden features output by the encoder. In this work, we focus on leveraging soft-probabilities in a black-box framework, i.e. we can access the soft-predictions but not the internal states of the model. Our contributions include: (i) RAINPROOF a Relative informAItioN Projection OOD detection framework; and (ii) a more operational evaluation setting for OOD detection. Surprisingly, we find that OOD detection is not necessarily aligned with task-specific measures. The OOD detector may filter out samples well processed by the model and keep samples that are not, leading to weaker performance. Our results show that RAINPROOF provides OOD detection methods more aligned with task-specific performance metrics than traditional OOD detectors.