Point, Revise, Review: Grounded Agentic Analysis in Reactive Notebooks with marimo-lens
On the map: LLM agents and tool use → Web, search, and research agents also fits: Multimodal and embodied agents
Abstract (arXiv)
In a computational notebook, a human can point to a rendered result and ask about "this," while an agent acts through cells, dependencies, and runtime state. When what the human sees and what the agent operates on are disconnected, the human must describe what they mean and trace what the agent did in prose that strips away situated visual and computational context. We present marimo-lens, an extension to the reactive Python notebook marimo for grounded human-agent analysis. Lens connects a human's marked output and note to its producing cell and relevant contributing computation, giving the agent computational context for the request. It surfaces agent activity, returns selected results to the notebook, and preserves the initiating selection for human review and reopening. We illustrate the lifecycle through an exploratory human-agent analysis of a real-world open dataset, showing how visually situated questions lead to computational inspection, notebook action, and returned evidence for human review.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 32 references that are in the map. First posted on arXiv 17 Sep 2026.
Builds on
15 of 32 references.
References · 15
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 160
- 2024WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualizationcited by 38
- 2023DirectGPT: A Direct Manipulation Interface to Interact with Large Language Modelscited by 32
- 2023On the Design of AI-powered Code Assistants for Notebookscited by 28
- 2025ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editingcited by 28
- 2025Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systemscited by 12
- 2025From Charts to Code: A Hierarchical Benchmark for Multimodal Modelscited by 11
- 2024Deconstructing Human-AI Collaboration: Agency, Interaction, and Adaptationcited by 8
- 2026ChartEditBench: Evaluating Grounded Multi-Turn Chart Editing in Multimodal Language Modelscited by 6
- 2024A Deixis-Centered Approach for Documenting Remote Synchronous Communication around Data Visualizationscited by 5
- 2025A Scoping Review of Mixed Initiative Visual Analytics in the Automation Renaissancecited by 2
3 more
- 2025NoteEx: Interactive Visual Context Manipulation for LLM-Assisted Exploratory Data Analysis in Computational Notebookscited by 2
- 2026ChartREG++: Towards Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referringcited by 2
- 2026VACP: Visual Analytics Context Protocolcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ATP-Bench: Towards Agentic Tool Planning for MLLM Interleaved Generation
- 2025PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature
- 2026HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualization
- 2026GroundEval: A Deterministic Replacement for LLM-as-Judge in Stateful Agent Evaluation
- 2026Read the Paper, Write the Code: Agentic Reproduction of Social-Science Results
- 2026AgentSim: A Platform for Verifiable Agent-Trace Simulation
- 2026PaperClaw: Harnessing Agents for Autonomous Research and Human-in-the-Loop Refinement
- 2025Agent-Testing Agent: A Meta-Agent for Automated Testing and Evaluation of Conversational AI Agents