# CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Models

arXiv: https://arxiv.org/abs/2304.07366 · HTML: https://mlsanity.com/paper/2304.07366/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2023.

## Built on by (most cited first, 30 of 34)

- ['Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants](https://mlsanity.com/paper/2409.19430/) (2024)
- [Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulation](https://mlsanity.com/paper/2502.02780/) (2025)
- [Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent](https://mlsanity.com/paper/2502.11267/) (2025)
- [Not Just Novelty: A Longitudinal Study on Utility and Customization of an AI Workflow](https://mlsanity.com/paper/2402.09894/) (2024)
- [HICode: Hierarchical Inductive Coding with LLMs](https://mlsanity.com/paper/2509.17946/) (2025)
- [Automating Thematic Analysis: How LLMs Analyse Controversial Topics](https://mlsanity.com/paper/2405.06919/) (2024)
- [UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents](https://mlsanity.com/paper/2504.09407/) (2025)
- [How to Interpret Agent Behavior](https://mlsanity.com/paper/2605.13625/) (2026)
- [From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis](https://mlsanity.com/paper/2502.11919/) (2025)
- [Harnessing ChatGPT for thematic analysis: Are we ready?](https://mlsanity.com/paper/2310.14545/) (2023)
- [Thematic Analysis with Open-Source Generative AI and Machine Learning: A New Method for Inductive Qualitative Codebook Development](https://mlsanity.com/paper/2410.03721/) (2024)
- [The Evolution of LLM Adoption in Industry Data Curation Practices](https://mlsanity.com/paper/2412.16089/) (2024)
- [LOGOS: LLM-driven End-to-End Grounded Theory Development and Schema Induction for Qualitative Research](https://mlsanity.com/paper/2509.24294/) (2025)
- [Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering](https://mlsanity.com/paper/2510.12925/) (2025)
- [Large Language Model for Qualitative Research -- A Systematic Mapping Study](https://mlsanity.com/paper/2411.14473/) (2024)
- [Deconstructing Depression Stigma: Integrating AI-driven Data Collection and Analysis with Causal Knowledge Graphs](https://mlsanity.com/paper/2502.06075/) (2025)
- [Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry](https://mlsanity.com/paper/2510.25595/) (2025)
- [Reflexis: Supporting Reflexivity and Rigor in Collaborative Qualitative Analysis through Design for Deliberation](https://mlsanity.com/paper/2601.15445/) (2026)
- [Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis](https://mlsanity.com/paper/2405.05758/) (2024)
- [Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspective](https://mlsanity.com/paper/2504.10918/) (2025)
- [Agent-as-Peer-Debriefer: A Multi-Agent Framework with Perspective-Based Refinement for Qualitative Analysis](https://mlsanity.com/paper/2605.24600/) (2026)
- [greCAPTCHA: Assessing Understanding as Evidence of Research Authorship Under Generative AI](https://mlsanity.com/paper/2609.20481/) (2026)
- [Effects of Collaboration on the Performance of Interactive Theme Discovery Systems](https://mlsanity.com/paper/2408.09030/) (2024)
- [Assessing the Reliability of Large Language Models for Deductive Qualitative Coding: A Comparative Study of ChatGPT Interventions](https://mlsanity.com/paper/2507.14384/) (2025)
- [Large Language Models for Large-Scale, Rigorous Qualitative Analysis in Applied Health Services Research](https://mlsanity.com/paper/2601.14478/) (2026)
- [Designing Computational Tools for Exploring Causal Relationships in Qualitative Data](https://mlsanity.com/paper/2602.06506/) (2026)
- [Can LLMs Understand the Impact of Trauma? Costs and Benefits of LLMs Coding the Interviews of Firearm Violence Survivors](https://mlsanity.com/paper/2604.16132/) (2026)
- [CentaurTA Studio: A Self-Improving Human-Agent Collaboration System for Thematic Analysis](https://mlsanity.com/paper/2604.18589/) (2026)
- [LLMs in Qualitative Research: Opportunities, Limitations, and Practical Considerations](https://mlsanity.com/paper/2605.16538/) (2026)
- [Algorithmic Fragility and Persona Bias in LLM-Generated Autistic Communication](https://mlsanity.com/paper/2605.26397/) (2026)

## Abstract (arXiv)

Collaborative Qualitative Analysis (CQA) can enhance qualitative analysis rigor and depth by incorporating varied viewpoints. Nevertheless, ensuring a rigorous CQA procedure itself can be both demanding and costly. To lower this bar, we take a theoretical perspective to design the CollabCoder workflow, that integrates Large Language Models (LLMs) into key inductive CQA stages: independent open coding, iterative discussions, and final codebook creation. In the open coding phase, CollabCoder offers AI-generated code suggestions and records decision-making data. During discussions, it promotes mutual understanding by sharing this data within the coding team and using quantitative metrics to identify coding (dis)agreements, aiding in consensus-building. In the code grouping stage, CollabCoder provides primary code group suggestions, lightening the cognitive load of finalizing the codebook. A 16-user evaluation confirmed the effectiveness of CollabCoder, demonstrating its advantages over existing software and providing empirical insights into the role of LLMs in the CQA practice.