# ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agent

arXiv: https://arxiv.org/abs/2507.18165 · HTML: https://mlsanity.com/paper/2507.18165/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 35 of its 81 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 30 of 35)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [UFO: A UI-Focused Agent for Windows OS Interaction](https://mlsanity.com/paper/2402.07939/) (2024)
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://mlsanity.com/paper/2203.06566/) (2022)
- [AutoWebGLM: A Large Language Model-based Web Navigating Agent](https://mlsanity.com/paper/2404.03648/) (2024)
- [The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development](https://mlsanity.com/paper/2302.07080/) (2023)
- [The Metacognitive Demands and Opportunities of Generative AI](https://mlsanity.com/paper/2312.10893/) (2023)
- [Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance](https://mlsanity.com/paper/2410.12361/) (2024)
- [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023)
- [ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Language](https://mlsanity.com/paper/2311.01920/) (2023)
- [A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects](https://mlsanity.com/paper/2305.02750/) (2023)
- [Ask-before-Plan: Proactive Language Agents for Real-World Planning](https://mlsanity.com/paper/2406.12639/) (2024)
- [Towards Human-centered Proactive Conversational Agents](https://mlsanity.com/paper/2404.12670/) (2024)
- [Need Help? Designing Proactive AI Assistants for Programming](https://mlsanity.com/paper/2410.04596/) (2024)
- [Proactive Conversational Agents with Inner Thoughts](https://mlsanity.com/paper/2501.00383/) (2024)
- [Towards Uncertainty-Aware Language Agent](https://mlsanity.com/paper/2401.14016/) (2024)
- [Dual-View Visual Contextualization for Web Navigation](https://mlsanity.com/paper/2402.04476/) (2024)
- [LEVA: Using Large Language Models to Enhance Visual Analytics](https://mlsanity.com/paper/2403.05816/) (2024)
- [ComPeer: A Generative Conversational Agent for Proactive Peer Support](https://mlsanity.com/paper/2407.18064/) (2024)
- [AVA: Towards Autonomous Visualization Agents through Visual Perception-Driven Decision-Making](https://mlsanity.com/paper/2312.04494/) (2023)
- [LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Execution](https://mlsanity.com/paper/2411.05651/) (2024)
- [The Role of Inclusion, Control, and Ownership in Workplace AI-Mediated Communication](https://mlsanity.com/paper/2309.11599/) (2023)
- [InsightLens: Augmenting LLM-Powered Data Analysis with Interactive Insight Management and Navigation](https://mlsanity.com/paper/2404.01644/) (2024)
- [MLDT: Multi-Level Decomposition for Complex Long-Horizon Robotic Task Planning with Open-Source Large Language Model](https://mlsanity.com/paper/2403.18760/) (2024)
- [DataSite: Proactive Visual Data Exploration with Computation of Insight-based Recommendations](https://mlsanity.com/paper/1802.08621/) (2018)
- [Fine-Tuned Large Language Model for Visualization System: A Study on Self-Regulated Learning in Education](https://mlsanity.com/paper/2407.20570/) (2024)
- [InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computational Notebooks](https://mlsanity.com/paper/2307.07922/) (2023)
- [PhenoFlow: A Human-LLM Driven Visual Analytics System for Exploring Large and Complex Stroke Datasets](https://mlsanity.com/paper/2407.16329/) (2024)
- [SummAct: Uncovering User Intentions Through Interactive Behaviour Summarisation](https://mlsanity.com/paper/2410.08356/) (2024)
- [Malleable Overview-Detail Interfaces](https://mlsanity.com/paper/2503.07782/) (2025)

## Built on by (most cited first, 14 of 14)

- [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025)
- [Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agents](https://mlsanity.com/paper/2510.19771/) (2025)
- [Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Models](https://mlsanity.com/paper/2602.02039/) (2026)
- [GUIDE: A Benchmark for Understanding and Assisting Users in Open-Ended GUI Tasks](https://mlsanity.com/paper/2603.25864/) (2026)
- [TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualization](https://mlsanity.com/paper/2603.25063/) (2026)
- [ProEvent: An Event-centric Benchmark for Proactive Agents](https://mlsanity.com/paper/2607.17701/) (2026)
- [Agents in the Large: Perception-Centered Architecture for Persistent Agents](https://mlsanity.com/paper/2608.30478/) (2026)
- [Beyond Chat and Clicks: GUI Agents for In-Situ Assistance via Live Interface Transformation](https://mlsanity.com/paper/2604.14668/) (2026)
- [Spatula: Exploring On-Demand In-Situ Interfaces and Interaction for Attribute Control](https://mlsanity.com/paper/2607.10405/) (2026)
- [AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysis](https://mlsanity.com/paper/2608.17834/) (2026)
- [From Question-First to Analyst-First: Domain-Expert Skills and Verified Knowledge Compilation for Proactive Enterprise Analytics](https://mlsanity.com/paper/2608.28594/) (2026)
- [WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics](https://mlsanity.com/paper/2609.24094/) (2026)
- [DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard Analysis](https://mlsanity.com/paper/2609.32385/) (2026)
- [Beyond End-to-End Black Box Mapping: An Intentional Agent Framework for Cognitive-driven Facial Reaction Generation](https://mlsanity.com/paper/2609.34419/) (2026)

## Abstract (arXiv)

Visual analytics (VA) is typically applied to complex data, thus requiring complex tools. While visual analytics empowers analysts in data analysis, analysts may get lost in the complexity occasionally. This highlights the need for intelligent assistance mechanisms. However, even the latest LLM-assisted VA systems only provide help when explicitly requested by the user, making them insufficiently intelligent to offer suggestions when analysts need them the most. We propose a ProactiveVA framework in which LLM-powered UI agent monitors user interactions and delivers context-aware assistance proactively. To design effective proactive assistance, we first conducted a formative study analyzing help-seeking behaviors in user interaction logs, identifying when users need proactive help, what assistance they require, and how the agent should intervene. Based on this analysis, we distilled key design requirements in terms of intent recognition, solution generation, interpretability and controllability. Guided by these requirements, we develop a three-stage UI agent pipeline including perception, reasoning, and acting. The agent autonomously perceives users' needs from VA interaction logs, providing tailored suggestions and intuitive guidance through interactive exploration of the system. We implemented the framework in two representative types of VA systems, demonstrating its generalizability, and evaluated the effectiveness through an algorithm evaluation, case and expert study and a user study. We also discuss current design trade-offs of proactive VA and areas for further exploration.