# How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study

arXiv: https://arxiv.org/abs/2309.10108 · HTML: https://mlsanity.com/paper/2309.10108/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2023.

## Built on by (most cited first, 24 of 24)

- [Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forward](https://mlsanity.com/paper/2406.07765/) (2024)
- [WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization](https://mlsanity.com/paper/2408.01703/) (2024)
- [Data-driven Discovery with Large Generative Models](https://mlsanity.com/paper/2402.13610/) (2024)
- [Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decomposition](https://mlsanity.com/paper/2407.02651/) (2024)
- [Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learning](https://mlsanity.com/paper/2410.08922/) (2024)
- [Can GPT-4 Replicate Empirical Software Engineering Research?](https://mlsanity.com/paper/2310.01727/) (2023)
- [Data Analysis in the Era of Generative AI](https://mlsanity.com/paper/2409.18475/) (2024)
- [Human-AI Experience in Integrated Development Environments: A Systematic Literature Review](https://mlsanity.com/paper/2503.06195/) (2025)
- ["The Diagram is like Guardrails": Structuring GenAI-assisted Hypotheses Exploration with an Interactive Shared Representation](https://mlsanity.com/paper/2503.16791/) (2025)
- [BIASINSPECTOR: Detecting Bias in Structured Data through LLM Agents](https://mlsanity.com/paper/2504.04855/) (2025)
- [Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questions](https://mlsanity.com/paper/2507.21285/) (2025)
- [Large Language Models as Co-Pilots for Causal Inference in Medical Studies](https://mlsanity.com/paper/2407.19118/) (2024)
- ["Impressively Scary:" Exploring User Perceptions and Reactions to Unraveling Machine Learning Models in Social Media Applications](https://mlsanity.com/paper/2503.03927/) (2025)
- [Guidance Source Matters: How Guidance from AI, Expert, or a Group of Analysts Impacts Visual Data Preparation and Analysis](https://mlsanity.com/paper/2502.00682/) (2025)
- [Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworks](https://mlsanity.com/paper/2507.10208/) (2025)
- [Understanding Mental States in Active and Autonomous Driving with EEG](https://mlsanity.com/paper/2512.09190/) (2025)
- [Power Echoes: Investigating Moderation Biases in Online Power-Asymmetric Conflicts](https://mlsanity.com/paper/2603.01457/) (2026)
- [AI in the Workplace: The Impact of AI on Perceived Job Decency and Meaningfulness](https://mlsanity.com/paper/2605.28680/) (2026)
- [The New Social Image: How AI Competency and AI Proactivity Influence Self- and Peer-Perceptions in the Workplace](https://mlsanity.com/paper/2606.00182/) (2026)
- [AI Adoption Across a Multinational Workforce: Sociotechnical Conditions for GenAI Acceptance in Human Resources](https://mlsanity.com/paper/2606.17887/) (2026)
- [From Overload to Insights: How AI Agents Can Support Scientists in Analyzing Complex Data](https://mlsanity.com/paper/2607.16845/) (2026)
- [AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysis](https://mlsanity.com/paper/2608.17834/) (2026)
- [AI Soccer Analyst: Stage-Aware and Verifiable Human-AI Collaboration for Soccer Data Analysis](https://mlsanity.com/paper/2609.11224/) (2026)
- [ForkSCOPE: Charting the Agentic Garden of Forking Paths](https://mlsanity.com/paper/2609.12438/) (2026)

## Abstract (arXiv)

Data analysis is challenging as analysts must navigate nuanced decisions that may yield divergent conclusions. AI assistants have the potential to support analysts in planning their analyses, enabling more robust decision making. Though AI-based assistants that target code execution (e.g., Github Copilot) have received significant attention, limited research addresses assistance for both analysis execution and planning. In this work, we characterize helpful planning suggestions and their impacts on analysts' workflows. We first review the analysis planning literature and crowd-sourced analysis studies to categorize suggestion content. We then conduct a Wizard-of-Oz study (n=13) to observe analysts' preferences and reactions to planning assistance in a realistic scenario. Our findings highlight subtleties in contextual factors that impact suggestion helpfulness, emphasizing design implications for supporting different abstractions of assistance, forms of initiative, increased engagement, and alignment of goals between analysts and assistants.