# Using Large Language Models for Qualitative Analysis can Introduce Serious Bias

arXiv: https://arxiv.org/abs/2309.17147 · HTML: https://mlsanity.com/paper/2309.17147/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 77 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future](https://mlsanity.com/paper/2309.15402/) (2023)
- [Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks](https://mlsanity.com/paper/2306.07899/) (2023)
- [ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning](https://mlsanity.com/paper/2304.06588/) (2023)
- [Framework-Based Qualitative Analysis of Free Responses of Large Language Models: Algorithmic Fidelity](https://mlsanity.com/paper/2309.06364/) (2023)

## Built on by (most cited first, 22 of 22)

- [Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis](https://mlsanity.com/paper/2402.01386/) (2024)
- [Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI](https://mlsanity.com/paper/2409.09467/) (2024)
- [From Voices to Validity: Leveraging Large Language Models (LLMs) for Textual Analysis of Policy Stakeholder Interviews](https://mlsanity.com/paper/2312.01202/) (2023)
- [A Computational Method for Measuring "Open Codes" in Qualitative Analysis](https://mlsanity.com/paper/2411.12142/) (2024)
- [Deconstructing Depression Stigma: Integrating AI-driven Data Collection and Analysis with Causal Knowledge Graphs](https://mlsanity.com/paper/2502.06075/) (2025)
- [Expanding Horizons in HCI Research Through LLM-Driven Qualitative Analysis](https://mlsanity.com/paper/2401.04138/) (2024)
- [Using Instruction-Tuned Large Language Models to Identify Indicators of Vulnerability in Police Incident Narratives](https://mlsanity.com/paper/2412.11878/) (2024)
- [Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis](https://mlsanity.com/paper/2405.05758/) (2024)
- [LLM-Assisted Automated Deductive Coding of Dialogue Data: Leveraging Dialogue-Specific Characteristics to Enhance Contextual Understanding](https://mlsanity.com/paper/2504.19734/) (2025)
- [Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth](https://mlsanity.com/paper/2506.06991/) (2025)
- [Generative AI in Sociological Research: State of the Discipline](https://mlsanity.com/paper/2511.16884/) (2025)
- [Correct codes for the wrong reasons? validating LLMs as measurement instruments for theoretical constructs](https://mlsanity.com/paper/2606.28574/) (2026)
- [EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footage](https://mlsanity.com/paper/2607.06468/) (2026)
- [Computational Analysis of Character Development in Holocaust Testimonies](https://mlsanity.com/paper/2412.17063/) (2024)
- [Enhancing LLM-Based Data Annotation with Error Decomposition](https://mlsanity.com/paper/2601.11920/) (2026)
- [Cheap Learning: Maximising Performance of Language Models for Social Data Science Using Minimal Data](https://mlsanity.com/paper/2401.12295/) (2024)
- [Applying Large Language Models to Characterize Public Narratives](https://mlsanity.com/paper/2511.13505/) (2025)
- [Can machines perform a qualitative data analysis? Reading the debate with Alan Turing](https://mlsanity.com/paper/2512.04121/) (2025)
- [Can Large Language Models Replace Human Coders? Introducing ContentBench](https://mlsanity.com/paper/2602.19467/) (2026)
- [Can LLMs Capture Expert Uncertainty? A Comparative Analysis of Value Alignment in Ethnographic Qualitative Research](https://mlsanity.com/paper/2603.04897/) (2026)
- [Can Large Language Models Revolutionize Survey Research? Experiments with Disaster Preparedness Responses](https://mlsanity.com/paper/2605.19229/) (2026)
- [How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding](https://mlsanity.com/paper/2609.11109/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are quickly becoming ubiquitous, but the implications for social science research are not yet well understood. This paper asks whether LLMs can help us analyse large-N qualitative data from open-ended interviews, with an application to transcripts of interviews with Rohingya refugees in Cox's Bazaar, Bangladesh. We find that a great deal of caution is needed in using LLMs to annotate text as there is a risk of introducing biases that can lead to misleading inferences. We here mean bias in the technical sense, that the errors that LLMs make in annotating interview transcripts are not random with respect to the characteristics of the interview subjects. Training simpler supervised models on high-quality human annotations with flexible coding leads to less measurement error and bias than LLM annotations. Therefore, given that some high quality annotations are necessary in order to asses whether an LLM introduces bias, we argue that it is probably preferable to train a bespoke model on these annotations than it is to use an LLM for annotation.