mlsanityData to 7 Oct 2026

Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era

arXiv 2404.11457 ↗First posted 17 Apr 2024cs.IR cs.AI cs.CLcited by 97

Not on the map. Why?

Abstract (arXiv)

With the rapid advancements of large language models (LLMs), information retrieval (IR) systems, such as search engines and recommender systems, have undergone a significant paradigm shift. This evolution, while heralding new opportunities, introduces emerging challenges, particularly in terms of biases and unfairness, which may threaten the information ecosystem. In this paper, we present a comprehensive survey of existing works on emerging and pressing bias and unfairness issues in IR systems when the integration of LLMs. We first unify bias and unfairness issues as distribution mismatch problems, providing a groundwork for categorizing various mitigation strategies through distribution alignment. Subsequently, we systematically delve into the specific bias and unfairness issues arising from three critical stages of LLMs integration into IR systems: data collection, model development, and result evaluation. In doing so, we meticulously review and analyze recent literature, focusing on the definitions, characteristics, and corresponding mitigation strategies associated with these issues. Finally, we identify and highlight some open problems and challenges for future work, aiming to inspire researchers and stakeholders in the IR field and beyond to better understand and mitigate bias and unfairness issues of IR in this LLM era. We also consistently maintain a GitHub repository for the relevant papers and resources in this rising direction at https://github.com/KID-22/LLM-IR-Bias-Fairness-Survey.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 111 of its 181 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

111 of 181 references.

References · 111

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  10. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  11. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
38 more
  1. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  2. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  4. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  5. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  6. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  8. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  9. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  11. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  12. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  13. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  14. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  15. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  16. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  17. 2023A Survey on Large Language Models for Recommendationcited by 378
  18. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  19. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  20. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  21. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  22. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  23. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  24. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  25. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  26. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  27. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  28. 2020Bias and Debias in Recommender System: A Survey and Future Directionscited by 239
  29. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221
  30. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  31. 2018Gender Bias in Neural Natural Language Processingcited by 210
  32. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  33. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  34. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  35. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
  36. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 183
  37. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  38. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168

50 most cited of 111 All 111 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 26; 2025: 56; 2026: 15.

97 papers.

Sort:

By area:

  1. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 119
  2. 2024IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedbackcited by 40
  3. 2024Stereotype or Personalization? User Identity Biases Chatbot Recommendationscited by 33
  4. 2024LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMscited by 31
  5. 2024How Much Can RAG Help the Reasoning of LLM?cited by 31
  6. 2024Evaluating and Enhancing Large Language Models for Novelty Assessment in Scholarly Publicationscited by 26
  7. 2024Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systemscited by 26
  8. 2024AD-LLM: Benchmarking Large Language Models for Anomaly Detectioncited by 25
  9. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  10. 2024KGARevion: An AI Agent for Knowledge-Intensive Biomedical QAcited by 20
  11. 2024Agentic Information Retrievalcited by 19
  12. 2024SPRec: Self-Play to Debias LLM-based Recommendationcited by 18
38 more
  1. 2025Unmasking Conversational Bias in AI Multiagent Systemscited by 16
  2. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  3. 2025Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluationcited by 15
  4. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 13
  5. 2025Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Modelscited by 12
  6. 2025Do not Abstain! Identify and Solve the Uncertaintycited by 11
  7. 2025RecGPT Technical Reportcited by 11
  8. 2025FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Datacited by 10
  9. 2025Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviewscited by 10
  10. 2024No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Userscited by 9
  11. 2025Characterizing LLM-driven Social Network: The Chirper.ai Casecited by 9
  12. 2025Toward Safe and Human-Aligned Game Conversational Recommendation via Multi-Agent Decompositioncited by 9
  13. 2025Brevity is the soul of sustainability: Characterizing LLM response lengthscited by 9
  14. 2024Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agentscited by 8
  15. 2025FairEval: Evaluating Fairness in LLM-Based Recommendations with Personality Awarenesscited by 8
  16. 2025Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMscited by 8
  17. 2025BiasBusters: Uncovering and Mitigating Tool Selection Bias in Large Language Modelscited by 8
  18. 2024'Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMscited by 7
  19. 2025Addressing Bias in Generative AI: Challenges and Research Opportunities in Information Managementcited by 7
  20. 2025Towards Robust ESG Analysis Against Greenwashing Risks: Aspect-Action Analysis with Cross-Category Generalizationcited by 7
  21. 2026ProbeLLM: Automating Principled Diagnosis of LLM Failurescited by 7
  22. 2024Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AIcited by 6
  23. 2025CHOP: Mobile Operating Assistant with Constrained High-frequency Optimized Subtask Planningcited by 6
  24. 2024Exploring the Escalation of Source Bias in User, Data, and Recommender System Feedback Loopcited by 5
  25. 2024Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLMcited by 5
  26. 2025Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documentscited by 5
  27. 2025Comparing Moral Values in Western English-speaking societies and LLMs with Word Associationscited by 5
  28. 2025Whose Name Comes Up? Auditing LLM-Based Scholar Recommendationscited by 5
  29. 2025Evaluating Bias in Spoken Dialogue LLMs for Real-World Decisions and Recommendationscited by 5
  30. 2025Large Language Model Sourcing: A Surveycited by 5
  31. 2025Characterizing Web Search in The Age of Generative AIcited by 5
  32. 2024Post-hoc Study of Climate Microtargeting on Social Media Ads with LLMs: Thematic Insights and Fairness Evaluationcited by 4
  33. 2025Scalable and Ethical Insider Threat Detection through Data Synthesis and Analysis by LLMscited by 4
  34. 2025NExT-Search: Rebuilding User Feedback Ecosystem for Generative AI Searchcited by 4
  35. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  36. 2025EvalCards: A Framework for Standardized Evaluation Reportingcited by 4
  37. 2025Source Coverage and Citation Bias in LLM-based vs. Traditional Search Enginescited by 4
  38. 2026Retrieval Collapses When AI Pollutes the Webcited by 4

50 most cited of 97 All 97 (JSON)