Addressing Bias in Generative AI: Challenges and Research Opportunities in Information Management
Not on the map. Why?
Abstract (arXiv)
Generative AI technologies, particularly Large Language Models (LLMs), have transformed information management systems but introduced substantial biases that can compromise their effectiveness in informing business decision-making. This challenge presents information management scholars with a unique opportunity to advance the field by identifying and addressing these biases across extensive applications of LLMs. Building on the discussion on bias sources and current methods for detecting and mitigating bias, this paper seeks to identify gaps and opportunities for future research. By incorporating ethical considerations, policy implications, and sociotechnical perspectives, we focus on developing a framework that covers major stakeholders of Generative AI systems, proposing key research questions, and inspiring discussion. Our goal is to provide actionable pathways for researchers to address bias in LLM applications, thereby advancing research in information management that ultimately informs business practices. Our forward-looking framework and research agenda advocate interdisciplinary approaches, innovative methods, dynamic perspectives, and rigorous evaluation to ensure fairness and transparency in Generative AI-driven information systems. We expect this study to serve as a call to action for information management scholars to tackle this critical issue, guiding the improvement of fairness and effectiveness in LLM-based systems for business practice.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 35 of its 54 references that are in the map. First posted on arXiv 22 Jan 2025.
Builds on
35 of 54 references.
References · 35
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
- 2018Datasheets for Datasetscited by 1,386
- 2023A Survey on Multimodal Large Language Modelscited by 909
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
- 2017Counterfactual Fairnesscited by 804
- 2023Whose Opinions Do Language Models Reflect?cited by 773
- 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
- 2023Bias and Fairness in Large Language Models: A Surveycited by 639
- 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 384
23 more
- 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
- 2018Gender Bias in Neural Natural Language Processingcited by 211
- 2020Adversarial Filters of Dataset Biasescited by 160
- 2017Fairness in Machine Learning: Lessons from Political Philosophycited by 149
- 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 144
- 201850 Years of Test (Un)fairness: Lessons for Machine Learningcited by 101
- 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 98
- 2023Fast Model Debias with Machine Unlearningcited by 72
- 2023Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Taskscited by 51
- 2024The Ethics of ChatGPT in Medicine and Healthcare: A Systematic Review on Large Language Models (LLMs)cited by 45
- 2022Learning Fair Representation via Distributional Contrastive Disentanglementcited by 31
- 2022MABEL: Attenuating Gender Bias using Textual Entailment Datacited by 31
- 2024Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiencycited by 31
- 2021Evaluating Debiasing Techniques for Intersectional Biasescited by 30
- 2024The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiringcited by 29
- 2024Understanding Biases in ChatGPT-based Recommender Systems: Provider Fairness, Temporal Stability, and Recencycited by 28
- 2024Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?cited by 25
- 2023Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biasescited by 21
- 2023Evaluating Biased Attitude Associations of Language Models in an Intersectional Contextcited by 20
- 2023Bias and Fairness in Chatbots: An Overviewcited by 20
- 2024Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLPcited by 11
- 2023How Far Can It Go?: On Intrinsic Gender Bias Mitigation for Text Classificationcited by 10
- 2022An Analysis of Social Biases Present in BERT Variants Across Multiple Languagescited by 6
Built on by
Citing papers per year: 2025: 5; 2026: 2.
7 papers.
- 2026Large Language Model Reasoning Failurescited by 21
- 2025G-Sim: Generative Simulations with Large Language Models and Gradient-Free Calibrationcited by 8
- 2025Position: Bridge the Gaps between Machine Unlearning and AI Regulationcited by 7
- 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directionscited by 3
- 2025What Would an LLM Do? Evaluating Large Language Models for Policymaking to Alleviate Homelessnesscited by 1
- 2025Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanationscited by 1
- 2026Toward Governance-Aware Autonomous GIS: A Narrative Review of Ethical and Privacy Risks in LLM-Enabled GeoAIcited by 0
- 2026Toward Governance-Aware Autonomous GIS: A Narrative Review of Ethical and Privacy Risks in LLM-Enabled GeoAI14 Sep 2026
- 2026Large Language Model Reasoning Failures5 Feb 2026
- 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions29 Sep 2025
- 2025What Would an LLM Do? Evaluating Large Language Models for Policymaking to Alleviate Homelessness4 Sep 2025
- 2025Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations3 Sep 2025
- 2025G-Sim: Generative Simulations with Large Language Models and Gradient-Free Calibration10 Jun 2025
- 2025Position: Bridge the Gaps between Machine Unlearning and AI Regulation18 Feb 2025