mlsanityData to 14 Oct 2026

DRAMA: Unifying Data Retrieval and Analysis for Open-Domain Analytic Queries

arXiv 2510.27238 ↗First posted 31 Oct 2025cs.DB cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Manually conducting real-world data analyses is labor-intensive and inefficient. Despite numerous attempts to automate data science workflows, none of the existing paradigms or systems fully demonstrate all three key capabilities required to support them effectively: (1) open-domain data collection, (2) structured data transformation, and (3) analytic reasoning. To overcome these limitations, we propose DRAMA, an end-to-end paradigm that answers users' analytic queries in natural language on large-scale open-domain data. DRAMA unifies data collection, transformation, and analysis as a single pipeline. To quantitatively evaluate system performance on tasks representative of DRAMA, we construct a benchmark, DRAMA-Bench, consisting of two categories of tasks: claim verification and question answering, each comprising 100 instances. These tasks are derived from real-world applications that have gained significant public attention and require the retrieval and analysis of open-domain data. We develop DRAMA-Bot, a multi-agent system designed following DRAMA. It comprises a data retriever that collects and transforms data by coordinating the execution of sub-agents, and a data analyzer that performs structured reasoning over the retrieved data. We evaluate DRAMA-Bot on DRAMA-Bench together with five state-of-the-art baseline agents. DRAMA-Bot achieves 86.5% task accuracy at a cost of 0.05, outperforming all baselines with up to 6.9 times the accuracy and less than 1/6 of the cost. DRAMA is publicly available at https://github.com/uiuc-kang-lab/drama.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 33 of its 81 references that are in the map. First posted on arXiv 31 Oct 2025.

Builds on

33 of 81 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  6. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  7. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  8. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  9. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  10. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  11. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  12. 2023Evaluating Verifiability in Generative Search Enginescited by 303
21 more
  1. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  2. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  3. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  4. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 236
  5. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
  6. 2021Annotating and Modeling Fine-grained Factuality in Summarizationcited by 131
  7. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 126
  8. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 120
  9. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 97
  10. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 87
  11. 2023Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflowcited by 76
  12. 2020Understanding tables with intermediate pre-trainingcited by 68
  13. 2024TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarizationcited by 66
  14. 2024Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?cited by 56
  15. 2023KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Basescited by 52
  16. 2024DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processingcited by 52
  17. 2024A Declarative System for Optimizing AI Workloadscited by 37
  18. 2024Semantic Operators: A Declarative Model for Rich, AI-based Data Processingcited by 34
  19. 2024Text2SQL is Not Enough: Unifying AI and Databases with TAGcited by 34
  20. 2023Continual Learning for Generative Retrieval over Dynamic Corporacited by 25
  21. 2025LEAP: LLM-powered End-to-end Automatic Library for Processing Social Science Queries on Unstructured Datacited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Analytical Searchcited by 1