# Conversational Challenges in AI-Powered Data Science: Obstacles, Needs, and Design Opportunities

arXiv: https://arxiv.org/abs/2310.16164 · HTML: https://mlsanity.com/paper/2310.16164/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2023.

## Built on by (most cited first, 9 of 9)

- [WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization](https://mlsanity.com/paper/2408.01703/) (2024)
- [The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers](https://mlsanity.com/paper/2404.02806/) (2024)
- [Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decomposition](https://mlsanity.com/paper/2407.02651/) (2024)
- [What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use](https://mlsanity.com/paper/2409.08775/) (2024)
- [Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions](https://mlsanity.com/paper/2502.14202/) (2025)
- [DataLab: A Unified Platform for LLM-Powered Business Intelligence](https://mlsanity.com/paper/2412.02205/) (2024)
- [GPT in Data Science: A Practical Exploration of Model Selection](https://mlsanity.com/paper/2311.11516/) (2023)
- [Flowco: Rethinking Data Analysis in the Age of LLMs](https://mlsanity.com/paper/2504.14038/) (2025)
- [Open Tabular Insight Extraction: Where Do We Stand, and Where Should We Go?](https://mlsanity.com/paper/2609.07629/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are being increasingly employed in data science for tasks like data preprocessing and analytics. However, data scientists encounter substantial obstacles when conversing with LLM-powered chatbots and acting on their suggestions and answers. We conducted a mixed-methods study, including contextual observations, semi-structured interviews (n=14), and a survey (n=114), to identify these challenges. Our findings highlight key issues faced by data scientists, including contextual data retrieval, formulating prompts for complex tasks, adapting generated code to local environments, and refining prompts iteratively. Based on these insights, we propose actionable design recommendations, such as data brushing to support context selection, and inquisitive feedback loops to improve communications with AI-based assistants in data-science tools.