mlsanityData to 14 Oct 2026

Not Everything That Counts Can Be Counted: A Case for Safe Qualitative AI

arXiv 2511.09325 ↗First posted 12 Nov 2025cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence (AI) and large language models (LLM) are reshaping science, with most recent advances culminating in fully-automated scientific discovery pipelines. But qualitative research has been left behind. Researchers in qualitative methods are hesitant about AI adoption. Yet when they are willing to use AI at all, they have little choice but to rely on general-purpose tools like ChatGPT to assist with interview interpretation, data annotation, and topic modeling - while simultaneously acknowledging these system's well-known limitations of being biased, opaque, irreproducible, and privacy-compromising. This creates a critical gap: while AI has substantially advanced quantitative methods, the qualitative dimensions essential for meaning-making and comprehensive scientific understanding remain poorly integrated. We argue for developing dedicated qualitative AI systems built from the ground up for interpretive research. Such systems must be transparent, reproducible, and privacy-friendly. We review recent literature to show how existing automated discovery pipelines could be enhanced by robust qualitative capabilities, and identify key opportunities where safe qualitative AI could advance multidisciplinary and mixed-methods research.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 38 references that are in the map. First posted on arXiv 12 Nov 2025.

Builds on

11 of 38 references.

References · 11

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  4. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  5. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 422
  6. 2025Accelerating scientific discovery with Co-Scientistcited by 401
  7. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  8. 2023Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Codingcited by 82
  9. 2025Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?cited by 75
  10. 2025Chatbots for Data Collection in Surveys: A Comparison of Four Theory-Based Interview Probescited by 9
  11. 2023Harnessing ChatGPT for thematic analysis: Are we ready?cited by 8

Built on by

Not cited yet.