mlsanityData to 14 Oct 2026

Generating Expressive and Customizable Evals for Timeseries Data Analysis Agents with AgentFuel

arXiv 2603.12483 ↗First posted 12 Mar 2026cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Across many domains (e.g., IoT, observability, telecommunications, cybersecurity), there is an emerging adoption of conversational data analysis agents that enable users to "talk to your data" to extract insights. Such data analysis agents operate on timeseries data models; e.g., measurements from sensors or events monitoring user clicks and actions in product analytics. We evaluate 6 popular data analysis agents (both open-source and proprietary) on domain-specific data and query types, and find that they fail on stateful and incident-specific queries. We observe two key expressivity gaps in existing evals: domain-customized datasets and domain-specific query types. To enable practitioners in such domains to generate customized and expressive evals for such timeseries data agents, we present AgentFuel. AgentFuel helps domain experts quickly create customized evals to perform end-to-end functional tests. We show that AgentFuel's benchmarks expose key directions for improvement in existing data agent frameworks. We also present anecdotal evidence that using AgentFuel can improve agent performance (e.g., with GEPA). AgentFuel benchmarks are available at https://huggingface.co/datasets/RockfishData/TimeSeriesAgentEvals.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 40 of its 87 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

40 of 87 references.

References · 40

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  4. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
  5. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  6. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  7. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
  8. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  9. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  10. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
  11. 2025Survey on Evaluation of LLM-based Agentscited by 204
  12. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 152
28 more
  1. 2025MemOS: A Memory OS for AI Systemcited by 138
  2. 2024DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?cited by 125
  3. 2024Language Models Still Struggle to Zero-shot Reason about Time Seriescited by 88
  4. 2022Natural Language to Code Generation in Interactive Data Science Notebookscited by 87
  5. 2024Context is Key: A Benchmark for Forecasting with Essential Textual Informationcited by 87
  6. 2024BLADE: Benchmarking Language Model Agents for Data-Driven Sciencecited by 77
  7. 2026Towards a Science of AI Agent Reliabilitycited by 65
  8. 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Modelscited by 58
  9. 2025DataSciBench: An LLM Agent Benchmark for Data Sciencecited by 57
  10. 2024TimeSeriesExam: A time series understanding examcited by 49
  11. 2025Measuring Agents in Productioncited by 37
  12. 2025NL2SQL-BUGs: A Benchmark for Detecting Semantic Errors in NL2SQL Translationcited by 33
  13. 2025KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakescited by 33
  14. 2024Benchmarking Data Science Agentscited by 30
  15. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  16. 2025IntellAgent: A Multi-Agent Framework for Evaluating Conversational AI Systemscited by 25
  17. 2024Tapilot-Crossing: Benchmarking and Evolving LLMs Towards Interactive Data Analysis Agentscited by 17
  18. 2025Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Modelscited by 15
  19. 2026AgentSM: Semantic Memory for Agentic Text-to-SQLcited by 14
  20. 2026Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboardscited by 13
  21. 2025Knowledge Base Construction for Knowledge-Augmented Text-to-SQLcited by 9
  22. 2026Arming Data Agents with Tribal Knowledgecited by 7
  23. 2024XForecast: Evaluating Natural Language Explanations for Time Series Forecastingcited by 4
  24. 2026Insight Agents: An LLM-Based Multi-Agent System for Data Insightscited by 4
  25. 2025Text-to-SQL for Enterprise Data Analyticscited by 3
  26. 2025ConDABench: Interactive Evaluation of Language Models for Data Analysiscited by 3
  27. 2024Evaluating SQL Understanding in Large Language Modelscited by 2
  28. 2025Evaluating NL2SQL via SQL2NLcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Symbolic Separation: Grounding Deep Agents in Knowledge Graphs for Trustworthy Operational Data Analyticscited by 0