# Antipatterns in AI-assisted Qualitative Data Analysis: A Catalog of Temptations and Pitfalls for Software Engineering Researchers

arXiv: https://arxiv.org/abs/2608.27927 · HTML: https://mlsanity.com/paper/2608.27927/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 54 references that are in the map.
- First posted on arXiv 28 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Guidelines for Empirical Studies in Software Engineering involving Large Language Models](https://mlsanity.com/paper/2508.15503/) (2025)
- [Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis](https://mlsanity.com/paper/2402.01386/) (2024)
- [Efficient and Green Large Language Models for Software Engineering: Literature Review, Vision, and the Road Ahead](https://mlsanity.com/paper/2404.04566/) (2024)
- [Towards Evaluation Guidelines for Empirical Studies involving LLMs](https://mlsanity.com/paper/2411.07668/) (2024)
- [Generative AI and Empirical Software Engineering: A Paradigm Shift](https://mlsanity.com/paper/2502.08108/) (2025)
- [Applications and Implications of Large Language Models in Qualitative Analysis: A New Frontier for Empirical Software Engineering](https://mlsanity.com/paper/2412.06564/) (2024)
- [Socio-Technical Grounded Theory for Software Engineering](https://mlsanity.com/paper/2103.14235/) (2021)
- [LLM-Assisted Thematic Analysis: Opportunities, Limitations, and Recommendations](https://mlsanity.com/paper/2511.14528/) (2025)
- [Guiding Principles for Using Mixed Methods Research in Software Engineering](https://mlsanity.com/paper/2404.06011/) (2024)
- [Large Language Model for Qualitative Research -- A Systematic Mapping Study](https://mlsanity.com/paper/2411.14473/) (2024)
- [Get on the Train or be Left on the Station: Using LLMs for Software Engineering Research](https://mlsanity.com/paper/2506.12691/) (2025)
- [Generative Artificial Intelligence in Qualitative Research Methods: Between Hype and Risks?](https://mlsanity.com/paper/2511.08461/) (2025)
- [On the Use of Large Language Models for Qualitative Synthesis](https://mlsanity.com/paper/2510.16502/) (2025)
- [Qualitative Coding Analysis through Open-Source Large Language Models: A User Study and Design Recommendations](https://mlsanity.com/paper/2602.18352/) (2026)
- [Prompt Engineering Strategies for LLM-based Qualitative Coding of Psychological Safety in Software Engineering Communities: A Controlled Empirical Study](https://mlsanity.com/paper/2605.07422/) (2026)

## Abstract (arXiv)

AI-assisted qualitative data analysis (QDA) offers unprecedented opportunities to streamline software engineering (SE) research, yet uncritical use risks compromising analytical rigor and flooding the field with accelerated production of low-quality research. While tactical best practices will naturally evolve over time, SE researchers currently lack strategic guidance to identify and mitigate methodological risks when attempting AI-assisted QDA. Based on our decades of qualitative SE research expertise and experience combined with an understanding of the emerging landscape of AI-assisted QDA, this paper presents a catalog of antipatterns in AI-assisted QDA - a set of assumptions and practices that initially appear advantageous but ultimately undermine analytical rigor and validity. The antipatterns are grouped into three categories reflecting escalating impact: Dangerous Drivers, Operational Missteps, and Analytical Failures. As more SE researchers attempt AI-assisted QDA, these antipatterns will help them identify and avoid common temptations and pitfalls, while reviewers can be equipped with the vocabulary and criteria to call out problematic and failed practice. Ultimately, this catalog of antipatterns can serve as a stepping stone in our responsible methodological evolution toward principled and meaningful human-AI collaboration in qualitative research.