# H is for Human and How (Not) To Evaluate Qualitative Research in HCI

arXiv: https://arxiv.org/abs/2409.01302 · HTML: https://mlsanity.com/paper/2409.01302/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2024.

## Built on by (most cited first, 6 of 6)

- ['Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants](https://mlsanity.com/paper/2409.19430/) (2024)
- [The Case for "Thick Evaluations" of Cultural Representation in AI](https://mlsanity.com/paper/2503.19075/) (2025)
- [Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLM](https://mlsanity.com/paper/2410.14879/) (2024)
- [Human-Centered AI Communication in Co-Creativity: An Initial Framework and Insights](https://mlsanity.com/paper/2505.18385/) (2025)
- [Generative Artificial Intelligence in Qualitative Research Methods: Between Hype and Risks?](https://mlsanity.com/paper/2511.08461/) (2025)
- [Large language models struggle with ethnographic text annotation](https://mlsanity.com/paper/2601.12099/) (2026)

## Abstract (arXiv)

Concern has recently been expressed by HCI researchers as to the inappropriate treatment of qualitative studies through a positivistic mode of evaluation that places emphasis on metrics and measurement. This contrasts with the nature of qualitative research, which privileges interpretation and understanding over quantification. This paper explains the difference between positivism and interpretivism, the limits of quantification in human science, the distinctive contribution of qualitative research, and how quality assurance might be provided for in the absence of numbers via five basic criteria that reviewers may use to evaluate qualitative studies on their own terms.