# Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis

arXiv: https://arxiv.org/abs/2402.01386 · HTML: https://mlsanity.com/paper/2402.01386/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 27 of its 81 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 27 of 27)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [The Impact of AI on Developer Productivity: Evidence from GitHub Copilot](https://mlsanity.com/paper/2302.06590/) (2023)
- [A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT](https://mlsanity.com/paper/2303.04226/) (2023)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [Large Language Models for Software Engineering: Survey and Open Problems](https://mlsanity.com/paper/2310.03533/) (2023)
- [Software Testing with Large Language Models: Survey, Landscape, and Vision](https://mlsanity.com/paper/2307.07221/) (2023)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning](https://mlsanity.com/paper/2304.06588/) (2023)
- [Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding](https://mlsanity.com/paper/2304.10548/) (2023)
- [Automated Repair of Programs from Large Language Models](https://mlsanity.com/paper/2205.10583/) (2022)
- [LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis](https://mlsanity.com/paper/2310.15100/) (2023)
- [Towards an Understanding of Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2308.11396/) (2023)
- [LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding](https://mlsanity.com/paper/2306.14924/) (2023)
- [Towards Human-Bot Collaborative Software Architecting with ChatGPT](https://mlsanity.com/paper/2302.14600/) (2023)
- [Using Large Language Models for Qualitative Analysis can Introduce Serious Bias](https://mlsanity.com/paper/2309.17147/) (2023)
- [A/B Testing: A Systematic Literature Review](https://mlsanity.com/paper/2308.04929/) (2023)
- [Autonomous Agents in Software Development: A Vision Paper](https://mlsanity.com/paper/2311.18440/) (2023)
- [Understanding Emails and Drafting Responses -- An Approach Using GPT-3](https://mlsanity.com/paper/2102.03062/) (2021)
- [Navigating Complexity in Software Engineering: A Prototype for Comparing GPT-n Solutions](https://mlsanity.com/paper/2301.12169/) (2023)
- [Expanding Horizons in HCI Research Through LLM-Driven Qualitative Analysis](https://mlsanity.com/paper/2401.04138/) (2024)

## Built on by (most cited first, 20 of 20)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [Large Language Models for Manufacturing](https://mlsanity.com/paper/2410.21418/) (2024)
- [Large Language Model-based Data Science Agent: A Survey](https://mlsanity.com/paper/2508.02744/) (2025)
- [System for systematic literature review using multiple AI agents: Concept and an empirical evaluation](https://mlsanity.com/paper/2403.08399/) (2024)
- [Large Language Model Evaluation Via Multi AI Agents: Preliminary results](https://mlsanity.com/paper/2404.01023/) (2024)
- [DataSTORM: Deep Research on Large-Scale Databases using Exploratory Data Analysis and Data Storytelling](https://mlsanity.com/paper/2604.06474/) (2026)
- [Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI](https://mlsanity.com/paper/2411.08881/) (2024)
- [AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery](https://mlsanity.com/paper/2504.07421/) (2025)
- [POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Models](https://mlsanity.com/paper/2505.06579/) (2025)
- [Temperature and Persona Shape LLM Agent Consensus With Minimal Accuracy Gains in Qualitative Coding](https://mlsanity.com/paper/2507.11198/) (2025)
- [SFT-TA: Supervised Fine-Tuned Agents in Multi-Agent LLMs for Automated Inductive Thematic Analysis](https://mlsanity.com/paper/2509.17167/) (2025)
- [From Clicks to Conversations: Evaluating the Effectiveness of Conversational Agents in Statistical Analysis](https://mlsanity.com/paper/2502.08114/) (2025)
- [Distributed Approach to Haskell Based Applications Refactoring with LLMs Based Multi-Agent Systems](https://mlsanity.com/paper/2502.07928/) (2025)
- [Leveraging Interview-Informed LLMs to Model Survey Responses: Comparative Insights from AI-Generated and Human Data](https://mlsanity.com/paper/2505.21997/) (2025)
- [From Specification to Service: Accelerating API-First Development Using Multi-Agent Systems](https://mlsanity.com/paper/2510.19274/) (2025)
- [Disagreement as Data: Reasoning Trace Analytics in Multi-Agent Systems](https://mlsanity.com/paper/2601.12618/) (2026)
- [Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline](https://mlsanity.com/paper/2606.29014/) (2026)
- [Antipatterns in AI-assisted Qualitative Data Analysis: A Catalog of Temptations and Pitfalls for Software Engineering Researchers](https://mlsanity.com/paper/2608.27927/) (2026)
- [Federated Agent Optimization](https://mlsanity.com/paper/2610.01195/) (2026)

## Abstract (arXiv)

Context: Manual qualitative data analysis is time-intensive and can compromise validity and replicability, affecting analysis design, implementation, and reporting. Large Language Models (LLMs) enable human-bot collaboration in Software Engineering (SE), but their potential for qualitative data analysis in SE remains largely unexplored. Objective: The objective of this study is to design and develop an LLM-based multi-agent system that synergizes human decision support with AI to automate various qualitative data analysis approaches. Methods: We used LLM-based multi-agents systems to assist the qualitative data analysis process, deploying 27 agents, each responsible for a specific task, such as text summarization, initial code generation, and extracting themes and patterns. Results: The main findings are: (1) the LLM-based multi-agent system accelerates the qualitative data analysis process, (2) the system effectively automates tasks such as text summarization, initial code generation, and theme extraction, and (3) the publicly accessible code facilitates validation and further evaluation. Conclusion: The proposed LLM-based multi-agent system automates qualitative data analysis process, creating opportunities for researchers and practitioners. Future improvements focus on enhancing multilingual performance and integrating continuous expert feedback. The source code of proposed system and system details can be found here: https://github.com/GPT-Laboratory/Qualitative-Analysis-with-an-LLM-Based-Agentts