# From Clicks to Conversations: Evaluating the Effectiveness of Conversational Agents in Statistical Analysis

arXiv: https://arxiv.org/abs/2502.08114 · HTML: https://mlsanity.com/paper/2502.08114/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 8 of its 59 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Conceptual Metaphors Impact Perceptions of Human-AI Collaboration](https://mlsanity.com/paper/2008.02311/) (2020)
- [WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization](https://mlsanity.com/paper/2408.01703/) (2024)
- [Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis](https://mlsanity.com/paper/2402.01386/) (2024)
- [ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience](https://mlsanity.com/paper/2307.01135/) (2023)
- [ODSearch: Fast and Resource Efficient On-device Natural Language Search for Fitness Trackers' Data](https://mlsanity.com/paper/2201.13202/) (2022)
- [A critical analysis of cognitive load measurement methods for evaluating the usability of different types of interfaces: guidelines and framework for Human-Computer Interaction](https://mlsanity.com/paper/2402.11820/) (2024)
- [Can LLMs substitute SQL? Comparing Resource Utilization of Querying LLMs versus Traditional Relational Databases](https://mlsanity.com/paper/2404.08727/) (2024)

## Built on by (most cited first, 2 of 2)

- [Beyond touch-based human-machine interface: Control your machines in natural language by utilizing large language models and OPC UA](https://mlsanity.com/paper/2510.11300/) (2025)
- [Delegating or Doing? Understanding User Behavior in Hybrid Human-Agent Interfaces](https://mlsanity.com/paper/2608.19551/) (2026)

## Abstract (arXiv)

The rapid proliferation of data science forced different groups of individuals with different backgrounds to adapt to statistical analysis. We hypothesize that conversational agents are better suited for statistical analysis than traditional graphical user interfaces (GUI). In this work, we propose a novel conversational agent, StatZ, for statistical analysis. We evaluate the efficacy of StatZ relative to established statistical software:SPSS, SAS, Stata, and JMP in terms of accuracy, task completion time, user experience, and user satisfaction. We combined the proposed analysis question from state-of-the-art language models with suggestions from statistical analysis experts and tested with 51 participants from diverse backgrounds. Our experimental design assessed each participant's ability to perform statistical analysis tasks using traditional statistical analysis tools with GUI and our conversational agent. Results indicate that the proposed conversational agents significantly outperform GUI statistical software in all assessed metrics, including quantitative (task completion time, accuracy, and user experience), and qualitative (user satisfaction) metrics. Our findings underscore the potential of using conversational agents to enhance statistical analysis processes, reducing cognitive load and learning curves and thereby proliferating data analysis capabilities, to individuals with limited knowledge of statistics.