# WeaveData: A Multimodal Data Analysis System with Self-Critiquing and Self-Evolving LLM Plans

arXiv: https://arxiv.org/abs/2609.34764 · HTML: https://mlsanity.com/paper/2609.34764/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 10 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Placed in

- Synthetic data and self-improvement for LLMs

## Builds on (most cited first, 4 of 4)

- [CAESURA: Language Models as Multi-Modal Query Planners](https://mlsanity.com/paper/2308.03424/) (2023)
- [SemBench: A Benchmark for Semantic Query Processing Engines](https://mlsanity.com/paper/2511.01716/) (2025)
- [Beyond Relational: Semantic-Aware Multi-Modal Analytics with LLM-Native Query Optimization](https://mlsanity.com/paper/2511.19830/) (2025)
- [Towards Automated Cross-domain Exploratory Data Analysis through Large Language Models](https://mlsanity.com/paper/2412.07214/) (2024)

## Abstract (arXiv)

Multimodal data analysis, which answers questions over relational tables, text, and images, has attracted growing attention in the data management community. Large language models (LLMs) enable such analysis in natural language by generating analysis plans over relational and semantic operators. However, LLM-generated plans are error-prone: a plan may silently compute something other than what was asked, fail during execution, or return a result that misses the question. This paper presents WeaveData, a multimodal data analysis system with self-critiquing and self-evolving LLM plans. First, WeaveData generates a typed logical plan for each question and critiques it step by step before execution, and it checks the executed result against the question afterwards. Second, WeaveData evolves a plan that fails or misses the question: it diagnoses the failure with the actual data, reuses the results that remain valid, and accumulates planning experience for later questions. Third, WeaveData grounds planning in a metadata knowledge graph of all modalities, clarifies ambiguous questions with the user, and backs every model judgment with evidence in an interactive notebook. We demonstrate WeaveData on two public multimodal datasets.