# CoAIcoder: Examining the Effectiveness of AI-assisted Human-to-Human Collaboration in Qualitative Analysis

arXiv: https://arxiv.org/abs/2304.05560 · HTML: https://mlsanity.com/paper/2304.05560/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2023.

## Built on by (most cited first, 25 of 25)

- [Automating Thematic Analysis: How LLMs Analyse Controversial Topics](https://mlsanity.com/paper/2405.06919/) (2024)
- [Reflections on Inductive Thematic Saturation as a potential metric for measuring the validity of an inductive Thematic Analysis with LLMs](https://mlsanity.com/paper/2401.03239/) (2024)
- [A Computational Method for Measuring "Open Codes" in Qualitative Analysis](https://mlsanity.com/paper/2411.12142/) (2024)
- [LOGOS: LLM-driven End-to-End Grounded Theory Development and Schema Induction for Qualitative Research](https://mlsanity.com/paper/2509.24294/) (2025)
- [Human-AI Collaboration in Thematic Analysis using ChatGPT: A User Study and Design Recommendations](https://mlsanity.com/paper/2311.03999/) (2023)
- [Expanding Horizons in HCI Research Through LLM-Driven Qualitative Analysis](https://mlsanity.com/paper/2401.04138/) (2024)
- [Exploring Collaborative GenAI Agents in Synchronous Group Settings: Eliciting Team Perceptions and Design Considerations for the Future of Work](https://mlsanity.com/paper/2504.14779/) (2025)
- [Reflexis: Supporting Reflexivity and Rigor in Collaborative Qualitative Analysis through Design for Deliberation](https://mlsanity.com/paper/2601.15445/) (2026)
- [Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis](https://mlsanity.com/paper/2405.05758/) (2024)
- [Human-AI Collaborative Taxonomy Construction: A Case Study in Profession-Specific Writing Assistants](https://mlsanity.com/paper/2406.18675/) (2024)
- [Generative AI Tools in Academic Research: Applications and Implications for Qualitative and Quantitative Research Methodologies](https://mlsanity.com/paper/2408.06872/) (2024)
- [Generative Artificial Intelligence in Qualitative Research Methods: Between Hype and Risks?](https://mlsanity.com/paper/2511.08461/) (2025)
- [Easy Come, Easy Go? Examining the Perceptions and Learning Effects of LLM-based Chatbot in the Context of Search-as-Learning](https://mlsanity.com/paper/2410.01396/) (2024)
- [Processes Matter: How ML/GAI Approaches Could Support Open Qualitative Coding of Online Discourse Datasets](https://mlsanity.com/paper/2504.02887/) (2025)
- [Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspective](https://mlsanity.com/paper/2504.10918/) (2025)
- [Agent-as-Peer-Debriefer: A Multi-Agent Framework with Perspective-Based Refinement for Qualitative Analysis](https://mlsanity.com/paper/2605.24600/) (2026)
- [Effects of Collaboration on the Performance of Interactive Theme Discovery Systems](https://mlsanity.com/paper/2408.09030/) (2024)
- [LLMs Integration in Software Engineering Team Projects: Roles, Impact, and a Pedagogical Design Space for AI Tools in Computing Education](https://mlsanity.com/paper/2410.23069/) (2024)
- [Bridging Human Interpretation and Machine Representation: A Landscape of Qualitative Data Analysis in the LLM Era](https://mlsanity.com/paper/2601.11739/) (2026)
- [Designing Computational Tools for Exploring Causal Relationships in Qualitative Data](https://mlsanity.com/paper/2602.06506/) (2026)
- [Single-agent vs. Multi-agents for Automated Video Analysis of On-Screen Collaborative Learning Behaviors](https://mlsanity.com/paper/2604.03631/) (2026)
- [CentaurTA Studio: A Self-Improving Human-Agent Collaboration System for Thematic Analysis](https://mlsanity.com/paper/2604.18589/) (2026)
- [Co-Refine: AI-Powered Tool Supporting Qualitative Analysis](https://mlsanity.com/paper/2604.19309/) (2026)
- [How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding](https://mlsanity.com/paper/2609.11109/) (2026)
- [Can Vision-Language Models Analyze Human-Centered Video? Mapping Model Capabilities and Human-AI Collaborative Workflows](https://mlsanity.com/paper/2609.27327/) (2026)

## Abstract (arXiv)

While AI-assisted individual qualitative analysis has been substantially studied, AI-assisted collaborative qualitative analysis (CQA)-a process that involves multiple researchers working together to interpret data-remains relatively unexplored. After identifying CQA practices and design opportunities through formative interviews, we designed and implemented CoAIcoder, a tool leveraging AI to enhance human-to-human collaboration within CQA through four distinct collaboration methods. With a between-subject design, we evaluated CoAIcoder with 32 pairs of CQA-trained participants across common CQA phases under each collaboration method. Our findings suggest that while using a shared AI model as a mediator among coders could improve CQA efficiency and foster agreement more quickly in the early coding stage, it might affect the final code diversity. We also emphasize the need to consider the independence level when using AI to assist human-to-human collaboration in various CQA scenarios. Lastly, we suggest design implications for future AI-assisted CQA systems.