mlsanityData to 7 Oct 2026

How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?

arXiv 2404.03302 ↗First posted 4 Apr 2024cs.CLcited by 44

Not on the map. Why?

Abstract (arXiv)

By leveraging the retrieval of information from external knowledge databases, Large Language Models (LLMs) exhibit enhanced capabilities for accomplishing many knowledge-intensive tasks. However, due to the inherent flaws of current retrieval systems, there might exist irrelevant information within those retrieving top-ranked passages. In this work, we present a comprehensive investigation into the robustness of LLMs to different types of irrelevant information under various conditions. We initially introduce a framework to construct high-quality irrelevant information that ranges from semantically unrelated, partially related, and related to questions. Furthermore, our analysis demonstrates that the constructed irrelevant information not only scores highly on similarity metrics, being highly retrieved by existing systems, but also bears semantic connections to the context. Our investigation reveals that current LLMs still face challenges in discriminating highly semantically related information and can be easily distracted by these irrelevant yet misleading content. Besides, we also find that current solutions for handling irrelevant information have limitations in improving the robustness of LLMs to such distractions. All the resources are available on GitHub at https://github.com/Di-viner/LLM-Robustness-to-Irrelevant-Information.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 36 of its 40 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

36 of 40 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  10. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
24 more
  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  2. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  5. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  8. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  9. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  10. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  11. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  12. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  13. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  14. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  15. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  16. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  17. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  18. 2023Learning to Filter Context for Retrieval-Augmented Generationcited by 82
  19. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  20. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Modelcited by 32
  21. 2024Over-Reasoning and Redundant Calculation of Large Language Modelscited by 27
  22. 2024Deductive Beam Search: Decoding Deducible Rationale for Chain-of-Thought Reasoningcited by 24
  23. 2024E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Modelscited by 13
  24. 2024Instructing Large Language Models to Identify and Ignore Irrelevant Conditionscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 17; 2026: 10.

44 papers.

Sort:

By area:

  1. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  2. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generationcited by 78
  3. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systemscited by 69
  4. 2024KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generationcited by 67
  5. 2025ARM: Adaptive Reasoning Modelcited by 38
  6. 2024How Much Can RAG Help the Reasoning of LLM?cited by 31
  7. 2024Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Modelscited by 28
  8. 2024Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentationcited by 27
  9. 2025Taming Knowledge Conflicts in Language Modelscited by 27
  10. 2025How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmarkcited by 26
  11. 2024Making Long-Context Language Models Better Multi-Hop Reasonerscited by 25
  12. 2024Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Examplecited by 22
32 more
  1. 2024A Survey of Conversational Searchcited by 22
  2. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  3. 2024Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Modelscited by 14
  4. 2024E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoningcited by 13
  5. 2024RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answeringcited by 13
  6. 2024Evaluating the Performance and Robustness of LLMs in Materials Science Q&A and Property Predictionscited by 9
  7. 2025MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graphcited by 5
  8. 2025Thinking in a Crowd: How Auxiliary Information Shapes LLM Reasoningcited by 5
  9. 2024Compound-QA: A Benchmark for Evaluating LLMs on Compound Questionscited by 4
  10. 2025OpenRAG: Optimizing RAG End-to-End via In-Context Retrieval Learningcited by 4
  11. 2025Prompting Large Language Models with Partial Knowledge for Answering Questions with Unseen Entitiescited by 4
  12. 2025Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustnesscited by 4
  13. 2024Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learningcited by 3
  14. 2025Retrieval-Augmented Generation by Evidence Retroactivity in LLMscited by 3
  15. 2025Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generationcited by 3
  16. 2026CODA: Difficulty-Aware Compute Allocation for Adaptive Reasoningcited by 3
  17. 2026Don't Blindly Trust It: How Unreliable Feedback Breaks Tool-Using LLM Agentscited by 3
  18. 2024From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmapcited by 2
  19. 2025Evaluating the Retrieval Robustness of Large Language Modelscited by 2
  20. 2025Improving TCM Question Answering through Tree-Organized Self-Reflective Retrieval with LLMscited by 1
  21. 2025Context Selection and Rewriting for Video-based Educational Question Generationcited by 1
  22. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMscited by 1
  23. 2025Optimizing Retrieval for RAG via Reinforcement Learningcited by 1
  24. 2026CQC-RAG: Robust Retrieval-Augmented Generation via Cross-Query Consistencycited by 1
  25. 2025Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QAcited by 0
  26. 2026GRaSp: Automatic Example Optimization for In-Context Learning in Low-Data Taskscited by 0
  27. 2026Seirenes: Adversarial Self-Play with Evolving Distractions for LLM Reasoningcited by 0
  28. 2026CLORE: Content-Level Optimization for Reasoning Efficiencycited by 0
  29. 2026ACCORD: Action-Conditioned Contextual Grounding for Language Agentscited by 0
  30. 2026AI Assistants Overassistcited by 0
  31. 2026Agentic Scaffolding Amplifies Sycophantic Behavior in Large Language Modelscited by 0
  32. 2026Reading Too Much into Context: Passive Exposure Can Steer LLM Decisionscited by 0