# Enhancing Robustness in Large Language Models: Prompting for Mitigating the Impact of Irrelevant Information

arXiv: https://arxiv.org/abs/2408.10615 · HTML: https://mlsanity.com/paper/2408.10615/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 26 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [REFINER: Reasoning Feedback on Intermediate Representations](https://mlsanity.com/paper/2304.01904/) (2023)
- [Active Prompting with Chain-of-Thought for Large Language Models](https://mlsanity.com/paper/2302.12246/) (2023)
- [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022)
- [Answering Questions by Meta-Reasoning over Multiple Chains of Thought](https://mlsanity.com/paper/2304.13007/) (2023)
- [Self-Knowledge Guided Retrieval Augmentation for Large Language Models](https://mlsanity.com/paper/2310.05002/) (2023)
- [ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design](https://mlsanity.com/paper/2303.07839/) (2023)
- [Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models](https://mlsanity.com/paper/2302.00618/) (2023)
- [RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledge](https://mlsanity.com/paper/2311.08147/) (2023)
- [RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thought](https://mlsanity.com/paper/2305.11499/) (2023)
- [Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic](https://mlsanity.com/paper/2309.13339/) (2023)
- [Learning to Reason and Memorize with Self-Notes](https://mlsanity.com/paper/2305.00833/) (2023)
- [Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processes](https://mlsanity.com/paper/2301.01751/) (2023)

## Built on by (most cited first, 8 of 8)

- [Taming Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2503.10996/) (2025)
- [How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark](https://mlsanity.com/paper/2505.18761/) (2025)
- [Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Search](https://mlsanity.com/paper/2502.01609/) (2025)
- [Robust Reasoning Benchmark](https://mlsanity.com/paper/2604.08571/) (2026)
- [Does Prompt Design Impact Quality of Data Imputation by LLMs?](https://mlsanity.com/paper/2506.04172/) (2025)
- [Mitigating Context Interference for Reliable and Efficient Search Agents](https://mlsanity.com/paper/2608.10743/) (2026)
- [Automatic Prompt Generation via Adaptive Selection of Prompting Techniques](https://mlsanity.com/paper/2510.18162/) (2025)
- [Prompt Minimization: Reducing Input Redundancy Without Sacrificing Output Fidelity](https://mlsanity.com/paper/2609.31505/) (2026)

## Abstract (arXiv)

In recent years, Large language models (LLMs) have garnered significant attention due to their superior performance in complex reasoning tasks. However, recent studies may diminish their reasoning capabilities markedly when problem descriptions contain irrelevant information, even with the use of advanced prompting techniques. To further investigate this issue, a dataset of primary school mathematics problems containing irrelevant information, named GSMIR, was constructed. Testing prominent LLMs and prompting techniques on this dataset revealed that while LLMs can identify irrelevant information, they do not effectively mitigate the interference it causes once identified. A novel automatic construction method, ATF, which enhances the ability of LLMs to identify and self-mitigate the influence of irrelevant information, is proposed to address this shortcoming. This method operates in two steps: first, analysis of irrelevant information, followed by its filtering. The ATF method, as demonstrated by experimental results, significantly improves the reasoning performance of LLMs and prompting techniques, even in the presence of irrelevant information on the GSMIR dataset.