# TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness

arXiv: https://arxiv.org/abs/2512.01183 · HTML: https://mlsanity.com/paper/2512.01183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 1 Dec 2025.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 17 of 17)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [The Effect of Sampling Temperature on Problem Solving in Large Language Models](https://mlsanity.com/paper/2402.05201/) (2024)
- [Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training](https://mlsanity.com/paper/2405.20978/) (2024)
- [PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models](https://mlsanity.com/paper/2204.01321/) (2022)
- [Exploring the Impact of Temperature on Large Language Models:Hot or Cold?](https://mlsanity.com/paper/2506.07295/) (2025)
- [Optimizing Temperature for Language Models with Multi-Sample Inference](https://mlsanity.com/paper/2502.05234/) (2025)
- [One word at a time: adversarial attacks on retrieval models](https://mlsanity.com/paper/2008.02197/) (2020)
- [Natural Example-Based Explainability: a Survey](https://mlsanity.com/paper/2309.03234/) (2023)
- [RAGE Against the Machine: Retrieval-Augmented LLM Explanations](https://mlsanity.com/paper/2405.13000/) (2024)
- [Ranking-Incentivized Quality Preserving Content Modification](https://mlsanity.com/paper/2005.12989/) (2020)
- [Can GPT models Follow Human Summarization Guidelines? A Study for Targeted Communication Goals](https://mlsanity.com/paper/2310.16810/) (2023)
- [CREDENCE: Counterfactual Explanations for Document Ranking](https://mlsanity.com/paper/2302.04983/) (2023)

## Abstract (arXiv)

The evaluation of Retrieval-Augmented Generation (RAG) systems typically examines retrieval quality and generation parameters like temperature in isolation, overlooking their interaction. This work presents a systematic investigation of how text perturbations (simulating noisy retrieval) interact with temperature settings across multiple LLM runs. We propose a comprehensive RAG Perturbation-Temperature Analysis Framework that subjects retrieved documents to three distinct perturbation types across varying temperature settings. Through extensive experiments on HotpotQA with both open-source and proprietary LLMs, we demonstrate that performance degradation follows distinct patterns: high-temperature settings consistently amplify vulnerability to perturbations, while certain perturbation types exhibit non-linear sensitivity across the temperature range. Our work yields three key contributions: (1) a diagnostic benchmark for assessing RAG robustness, (2) an analytical framework for quantifying perturbation-temperature interactions, and (3) practical guidelines for model selection and parameter tuning under noisy retrieval conditions.