mlsanityData to 7 Oct 2026

Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses

arXiv 2609.35255 ↗First posted 28 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Large language model agents are increasingly deployed for data-intensive work, yet reliable data analysis requires more than general-purpose reasoning and ad hoc tool augmentation. Data Agents, equipped with workflow harnesses, offer a promising paradigm for automating the end-to-end data science lifecycle. This paper examines Data Agents from a harness-centric perspective. First, we introduce a taxonomy of Data Agents and associated data environments, organizing the literature around five functional stages: perception, planning, execution, verification, and repair. Second, we analyze the key technical routes within each stage, identifying 15 distinct approaches ranging from data structure probing to data state reconstruction. Third, we identify four open reliability problems: inactive semantic calibration, missing clarification, missing experience transfer, and the missing verification-repair repository. These problems explain why silent failures can persist even when individual components function correctly, highlighting the need for rigorous workflow harnesses and shared reliability resources. Finally, we summarize the horizontal task families of Data Agents, examine their vertical application settings, and benchmarks for evaluation, while maintaining a companion repository at https://github.com/DEEP-PolyU/Awesome-Data-Agents.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 129 of its 147 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

129 of 147 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026AgentRewind: Recoverable Execution for Long-Horizon LLM Agents14 Aug
  2. 2026REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces8 Jun
  3. 2026CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures25 May
  4. 2026A Self-Healing Framework for Reliable LLM-Based Autonomous Agents7 May
  5. 2026Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis27 Apr
  6. 2026AgentFixer: From Failure Detection to Fix Recommendations in LLM Agentic Systems18 Feb

Other references · 123

  1. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 153
  2. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 134
  3. 2024CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQLcited by 132
  4. 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 127
  5. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environmentscited by 120
  6. 2024MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualizationcited by 98
  7. 2025SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planningcited by 58
  8. 2025DataSciBench: An LLM Agent Benchmark for Data Sciencecited by 53
  9. 2025OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignmentcited by 51
  10. 2024DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processingcited by 49
  11. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 49
  12. 2025Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoningcited by 32
38 more
  1. 2023Entity Matching using Large Language Modelscited by 31
  2. 2023PDFTriage: Question Answering over Long, Structured Documentscited by 29
  3. 2024Evolutionary Large Language Model for Automated Feature Transformationcited by 29
  4. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 28
  5. 2023Agent-OM: Leveraging LLM Agents for Ontology Matchingcited by 26
  6. 2024DataNarrative: Automated Data-Driven Storytelling with Visualizations and Textscited by 25
  7. 2025Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewardscited by 25
  8. 2025AutoData: A Multi-Agent System for Open Web Data Collectioncited by 25
  9. 2026TimeART: Towards Agentic Time Series Reasoning via Tool-Augmentationcited by 25
  10. 2024A Survey on Large Language Model-based Agents for Statistics and Data Sciencecited by 22
  11. 2025AgenticData: An Agentic Data Analytics System for Heterogeneous Datacited by 22
  12. 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 22
  13. 2025DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecyclecited by 22
  14. 2025A Survey of Reasoning and Agentic Systems in Time Series with Large Language Modelscited by 21
  15. 2025MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Searchcited by 20
  16. 2024LLMClean: Context-Aware Tabular Data Cleaning via LLM-Generated OFDscited by 19
  17. 2025TableMind: An Autonomous Programmatic Agent for Tool-Augmented Table Reasoningcited by 19
  18. 2024UQE: A Query Engine for Unstructured Databasescited by 18
  19. 2025ToolFuzz -- Automated Agent Tool Testingcited by 18
  20. 2026ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruningcited by 18
  21. 2024Learning Evolving Tools for Large Language Modelscited by 15
  22. 2025LLM/Agent-as-Data-Analyst: A Surveycited by 15
  23. 2025CoDA: Agentic Systems for Collaborative Data Visualizationcited by 14
  24. 2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQLcited by 14
  25. 2025InterChat: Enhancing Generative Visual Analytics using Multimodal Interactionscited by 13
  26. 2025EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routingcited by 13
  27. 2025FDABench: A Benchmark for Data Agents on Analytical Queries over Heterogeneous Datacited by 13
  28. 2025Failure Makes the Agent Stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactionscited by 12
  29. 2024SiriusBI: A Comprehensive LLM-Powered Solution for Data Analytics in Business Intelligencecited by 11
  30. 2025IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysiscited by 11
  31. 2025ST-Raptor: LLM-Powered Semi-Structured Table Question Answeringcited by 11
  32. 2025Cortex AISQL: A Production SQL Engine for Unstructured Datacited by 11
  33. 2025Multi-Objective Agentic Rewrites for Unstructured Data Processingcited by 10
  34. 2026Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errorscited by 10
  35. 2026Can AI Agents Answer Your Data Questions? A Benchmark for Data Agentscited by 10
  36. 2024E2ETune: End-to-End Knob Tuning via Fine-tuned Generative Language Modelcited by 9
  37. 2024NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Languagecited by 9
  38. 2026Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMscited by 9

50 most cited of 123 All 123 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DS-Lighting: Making Agent Harnesses Explicit for Data-Science Automation
  2. 2026DSAgentBench: Can Agents Automate End-to-End Data-Science Workflows in Real Computer Environments?
  3. 2026Data Agents: Agentic Data Systems
  4. 2026MUSE: An Interactive Meta-Agent for Understanding and Steering LLM-powered Data Science Systems
  5. 2026Data Intelligence Agents: Interpreting, Modeling, and Querying Enterprise Data via Autonomous Coding Agents
  6. 2026AgenticDataBench: A Comprehensive Benchmark for Data Agents
  7. 2026Graph, Loop, and Harness Engineering for Zero-Trust Agentic Data Engineering and Analytical Processing
  8. 2025Data Agent: A Holistic Architecture for Orchestrating Data+AI Ecosystems