LLM-Assisted Thematic Analysis: Opportunities, Limitations, and Recommendations
Not on the map. Why?
Abstract (arXiv)
[Context] Large Language Models (LLMs) are increasingly used to assist qualitative research in Software Engineering (SE), yet the methodological implications of this usage remain underexplored. Their integration into interpretive processes such as thematic analysis raises fundamental questions about rigor, transparency, and researcher agency. [Objective] This study investigates how experienced SE researchers conceptualize the opportunities, risks, and methodological implications of integrating LLMs into thematic analysis. [Method] A reflective workshop with 25 ISERN researchers guided participants through structured discussions of LLM-assisted open coding, theme generation, and theme reviewing, using color-coded canvases to document perceived opportunities, limitations, and recommendations. [Results] Participants recognized potential efficiency and scalability gains, but highlighted risks related to bias, contextual loss, reproducibility, and the rapid evolution of LLMs. They also emphasized the need for prompting literacy and continuous human oversight. [Conclusion] Findings portray LLMs as tools that can support, but not substitute, interpretive analysis. The study contributes to ongoing community reflections on how LLMs can responsibly enhance qualitative research in SE.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Nov 2025.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Taking a Pulse on How Generative AI is Reshaping the Software Engineering Research Landscapecited by 0
- 2026A Research Agenda on Agents and Software Engineering: Outcomes from the Rio A2SE Seminarcited by 0
- 2026Can Large Language Models Reliably Code Qualitative Humanitarian Data? A Benchmark Study Against Human Expert Adjudicationcited by 0
- 2026Antipatterns in AI-assisted Qualitative Data Analysis: A Catalog of Temptations and Pitfalls for Software Engineering Researcherscited by 0
- 2026Designing an Auditable LLM-Supported Workflow for Qualitative Thematic Analysiscited by 0
- 2026Designing an Auditable LLM-Supported Workflow for Qualitative Thematic Analysis31 Aug 2026
- 2026Antipatterns in AI-assisted Qualitative Data Analysis: A Catalog of Temptations and Pitfalls for Software Engineering Researchers28 Aug 2026
- 2026Can Large Language Models Reliably Code Qualitative Humanitarian Data? A Benchmark Study Against Human Expert Adjudication25 Jun 2026
- 2026A Research Agenda on Agents and Software Engineering: Outcomes from the Rio A2SE Seminar12 May 2026
- 2026Taking a Pulse on How Generative AI is Reshaping the Software Engineering Research Landscape13 Apr 2026