mlsanityData to 7 Oct 2026

Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding

arXiv 2304.10548 ↗First posted 17 Apr 2023cs.CL cs.AI cs.HCcited by 82

Not on the map. Why?

Abstract (arXiv)

Qualitative analysis of textual contents unpacks rich and valuable information by assigning labels to the data. However, this process is often labor-intensive, particularly when working with large datasets. While recent AI-based tools demonstrate utility, researchers may not have readily available AI resources and expertise, let alone be challenged by the limited generalizability of those task-specific models. In this study, we explored the use of large language models (LLMs) in supporting deductive coding, a major category of qualitative analysis where researchers use pre-determined codebooks to label the data into a fixed set of codes. Instead of training task-specific models, a pre-trained LLM could be used directly for various tasks without fine-tuning through prompt learning. Using a curiosity-driven questions coding task as a case study, we found, by combining GPT-3 with expert-drafted codebooks, our proposed approach achieved fair to substantial agreements with expert-coded results. We lay out challenges and opportunities in using LLMs to support qualitative coding and beyond.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 7 of its 17 references that are in the map. First posted on arXiv 17 Apr 2023.

Builds on

7 of 17 references.

References · 7

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  6. 2022Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generationcited by 17
  7. 2022Conversational agents for fostering curiosity-driven learning in childrencited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 24; 2025: 24; 2026: 20.

82 papers.

Sort:

By area:

  1. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  2. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  3. 2024Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooMcited by 77
  4. 2023LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysiscited by 70
  5. 2023LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Codingcited by 64
  6. 2024'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participantscited by 27
  7. 2023Embedding Democratic Values into Social Media AIs via Societal Objective Functionscited by 25
  8. 2023LoFT: Local Proxy Fine-tuning For Improving Transferability Of Adversarial Attacks Against Large Language Modelcited by 21
  9. 2024Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysiscited by 20
  10. 2023Can Large Language Models emulate an inductive Thematic Analysis of semi-structured interviews? An exploration and provocation on the limits of the approach and the modelcited by 15
  11. 2025A Systematic Evaluation of LLM Strategies for Mental Health Text Analysis: Fine-tuning vs. Prompt Engineering vs. RAGcited by 14
  12. 2025HICode: Hierarchical Inductive Coding with LLMscited by 12
38 more
  1. 2024Automating Thematic Analysis: How LLMs Analyse Controversial Topicscited by 10
  2. 2024Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefitscited by 9
  3. 2024Exploring the Use of Abusive Generative AI Models on Civitaicited by 9
  4. 2024Interpreting Language Reward Models via Contrastive Explanationscited by 9
  5. 2025From Conversation to Orchestration: HCI Challenges and Opportunities in Interactive Multi-Agentic Systemscited by 9
  6. 2023Harnessing ChatGPT for thematic analysis: Are we ready?cited by 8
  7. 2023From Voices to Validity: Leveraging Large Language Models (LLMs) for Textual Analysis of Policy Stakeholder Interviewscited by 8
  8. 2024Predicting challenge moments from students' discourse: A comparison of GPT-4 to two traditional natural language processing approachescited by 8
  9. 2024QuaLLM: An LLM-based Framework to Extract Quantitative Insights from Online Forumscited by 8
  10. 2023Writing user personas with Large Language Models: Testing phase 6 of a Thematic Analysis of semi-structured interviewscited by 7
  11. 2023Can GPT-4 Replicate Empirical Software Engineering Research?cited by 7
  12. 2023Generative Artificial Intelligence in Learning Analytics: Contextualising Opportunities and Challenges through the Learning Analytics Cyclecited by 6
  13. 2024Reflections on Inductive Thematic Saturation as a potential metric for measuring the validity of an inductive Thematic Analysis with LLMscited by 6
  14. 2024A Computational Method for Measuring "Open Codes" in Qualitative Analysiscited by 6
  15. 2024The Evolution of LLM Adoption in Industry Data Curation Practicescited by 6
  16. 2024Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviorscited by 5
  17. 2025Labeling Free-text Data using Language Model Ensemblescited by 5
  18. 2024Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teachingcited by 4
  19. 2024Harnessing LLMs for Automated Video Content Analysis: An Exploratory Workflow of Short Videos on Depressioncited by 4
  20. 2024Post-hoc Study of Climate Microtargeting on Social Media Ads with LLMs: Thematic Insights and Fairness Evaluationcited by 4
  21. 2025Deconstructing Depression Stigma: Integrating AI-driven Data Collection and Analysis with Causal Knowledge Graphscited by 4
  22. 2025Text Annotation via Inductive Coding: Comparing Human Experts to LLMs in Qualitative Data Analysiscited by 4
  23. 2024Expanding Horizons in HCI Research Through LLM-Driven Qualitative Analysiscited by 3
  24. 2024Bridging Dictionary: AI-Generated Dictionary of Partisan Language Usecited by 3
  25. 2024Using Instruction-Tuned Large Language Models to Identify Indicators of Vulnerability in Police Incident Narrativescited by 3
  26. 2025Co-DETECT: Collaborative Discovery of Edge Cases in Text Classificationcited by 3
  27. 2025Temperature and Persona Shape LLM Agent Consensus With Minimal Accuracy Gains in Qualitative Codingcited by 3
  28. 2025Attention to Non-Adopterscited by 3
  29. 2026Reflexis: Supporting Reflexivity and Rigor in Collaborative Qualitative Analysis through Design for Deliberationcited by 3
  30. 2023Mapping the Challenges of HCI: An Application and Evaluation of ChatGPT for Mining Insights at Scalecited by 2
  31. 2024Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysiscited by 2
  32. 2024Human-AI Collaborative Taxonomy Construction: A Case Study in Profession-Specific Writing Assistantscited by 2
  33. 2024Automatic deductive coding in discourse analysis: an application of large language models in learning analyticscited by 2
  34. 2025Interview AI-ssistant: Designing for Real-Time Human-AI Collaboration in Interview Preparation and Executioncited by 2
  35. 2025LLM-Assisted Automated Deductive Coding of Dialogue Data: Leveraging Dialogue-Specific Characteristics to Enhance Contextual Understandingcited by 2
  36. 2025MuseScorer: Idea Originality Scoring At Scalecited by 2
  37. 2023Talk2Care: Facilitating Asynchronous Patient-Provider Communication with Large-Language-Modelcited by 1
  38. 2023OpenAi's GPT4 as coding assistantcited by 1

50 most cited of 82 All 82 (JSON)