# InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computational Notebooks

arXiv: https://arxiv.org/abs/2307.07922 · HTML: https://mlsanity.com/paper/2307.07922/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2023.

## Built on by (most cited first, 11 of 11)

- [Cocoa: Co-Planning and Co-Execution with AI Agents](https://mlsanity.com/paper/2412.10999/) (2024)
- [From Data to Story: Towards Automatic Animated Data Video Creation with LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2408.03876/) (2024)
- [ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agent](https://mlsanity.com/paper/2507.18165/) (2025)
- [Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration](https://mlsanity.com/paper/2309.15723/) (2023)
- [Data Analysis in the Era of Generative AI](https://mlsanity.com/paper/2409.18475/) (2024)
- [SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings](https://mlsanity.com/paper/2403.17784/) (2024)
- [DATAWEAVER: Authoring Data-Driven Narratives through the Integrated Composition of Visualization and Text](https://mlsanity.com/paper/2503.22946/) (2025)
- [LLM-Assisted Visual Analytics: Opportunities and Challenges](https://mlsanity.com/paper/2409.02691/) (2024)
- [ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMs](https://mlsanity.com/paper/2502.02329/) (2025)
- [Reflection on Data Storytelling Tools in the Generative AI Era from the Human-AI Collaboration Perspective](https://mlsanity.com/paper/2503.02631/) (2025)
- [Linking Heterogeneous Data with Coordinated Agent Flows for Social Media Analysis](https://mlsanity.com/paper/2510.26172/) (2025)

## Abstract (arXiv)

Computational notebooks have become increasingly popular for exploratory data analysis due to their ability to support data exploration and explanation within a single document. Effective documentation for explaining chart findings during the exploration process is essential as it helps recall and share data analysis. However, documenting chart findings remains a challenge due to its time-consuming and tedious nature. While existing automatic methods alleviate some of the burden on users, they often fail to cater to users' specific interests. In response to these limitations, we present InkSight, a mixed-initiative computational notebook plugin that generates finding documentation based on the user's intent. InkSight allows users to express their intent in specific data subsets through sketching atop visualizations intuitively. To facilitate this, we designed two types of sketches, i.e., open-path and closed-path sketch. Upon receiving a user's sketch, InkSight identifies the sketch type and corresponding selected data items. Subsequently, it filters data fact types based on the sketch and selected data items before employing existing automatic data fact recommendation algorithms to infer data facts. Using large language models (GPT-3.5), InkSight converts data facts into effective natural language documentation. Users can conveniently fine-tune the generated documentation within InkSight. A user study with 12 participants demonstrated the usability and effectiveness of InkSight in expressing user intent and facilitating chart finding documentation.