# From Static to Interactive: Authoring Interactive Visualizations via Natural Language

arXiv: https://arxiv.org/abs/2601.17736 · HTML: https://mlsanity.com/paper/2601.17736/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 61 references that are in the map.
- First posted on arXiv 25 Jan 2026.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://mlsanity.com/paper/1710.07300/) (2017)
- [Towards Natural Language Interfaces for Data Visualization: A Survey](https://mlsanity.com/paper/2109.03506/) (2021)
- [Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Models](https://mlsanity.com/paper/2302.02094/) (2023)
- [DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing](https://mlsanity.com/paper/2401.10880/) (2024)
- [Data Formulator: AI-powered Concept-driven Visualization Authoring](https://mlsanity.com/paper/2309.10094/) (2023)
- [Visualization Generation with Large Language Models: An Evaluation](https://mlsanity.com/paper/2401.11255/) (2024)
- [Towards Natural Language-Based Visualization Authoring](https://mlsanity.com/paper/2208.10947/) (2022)
- [How do you Converse with an Analytical Chatbot? Revisiting Gricean Maxims for Designing Analytical Conversational Behavior](https://mlsanity.com/paper/2203.08420/) (2022)
- [Beagle: Automated Extraction and Interpretation of Visualizations from the Web](https://mlsanity.com/paper/1711.05962/) (2017)
- [MoVer: Motion Verification for Motion Graphics Animations](https://mlsanity.com/paper/2502.13372/) (2025)
- [Converting Basic D3 Charts into Reusable Style Templates](https://mlsanity.com/paper/1609.05283/) (2016)
- [Augmenting Static Visualizations with PapARVis Designer](https://mlsanity.com/paper/2310.04826/) (2023)
- [DIVI: Dynamically Interactive Visualization](https://mlsanity.com/paper/2310.17814/) (2023)
- [SimVecVis: A Dataset for Enhancing MLLMs in Visualization Understanding](https://mlsanity.com/paper/2506.21319/) (2025)
- [VisAR: Bringing Interactivity to Static Data Visualizations through Augmented Reality](https://mlsanity.com/paper/1708.01377/) (2017)
- [Touching Data: A Discoverability-based Evaluation of a Visualization Interface for Tablet Computers](https://mlsanity.com/paper/1806.06084/) (2018)
- [A Spatial-Constraint Model for Manipulating Static Visualizations](https://mlsanity.com/paper/2303.14476/) (2023)

## Abstract (arXiv)

Interactivity is crucial for effective data visualizations. However, it is often challenging to implement interactions for existing static visualizations, since the underlying code and data for existing static visualizations are often not available, and it also takes significant time and effort to enable interactions for them even if the original code and data are available. To fill this gap, we propose Athanor, a novel approach to transform existing static visualizations into interactive ones using multimodal large language models (MLLMs) and natural language instructions. Our approach introduces three key innovations: (1) an action-modification interaction design space that maps visualization interactions into user actions and corresponding adjustments, (2) a multi-agent requirement analyzer that translates natural language instructions into an actionable operational space, and (3) a visualization abstraction transformer that converts static visualizations into flexible and interactive representations regardless of their underlying implementation. Athanor allows users to effortlessly author interactions through natural language instructions, eliminating the need for programming. We conducted two case studies and in-depth interviews with target users to evaluate our approach. The results demonstrate the effectiveness and usability of our approach in allowing users to conveniently enable flexible interactions for static visualizations.