# Automated Data Readiness for Scientific AI

arXiv: https://arxiv.org/abs/2607.02771 · HTML: https://mlsanity.com/paper/2607.02771/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Builds on (most cited first, 20 of 20)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [The Open Catalyst 2020 (OC20) Dataset and Community Challenges](https://mlsanity.com/paper/2010.09990/) (2020)
- [The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts](https://mlsanity.com/paper/2206.08917/) (2022)
- [Parsl: Pervasive Parallel Programming in Python](https://mlsanity.com/paper/1905.02158/) (2019)
- [Nature Language Model: Deciphering the Language of Nature for Scientific Discovery](https://mlsanity.com/paper/2502.07527/) (2025)
- [ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability](https://mlsanity.com/paper/2404.14712/) (2024)
- [Data Readiness for AI: A 360-Degree Survey](https://mlsanity.com/paper/2404.05779/) (2024)
- [Data Readiness Levels](https://mlsanity.com/paper/1705.02245/) (2017)
- [Applying the FAIR Principles to computational workflows](https://mlsanity.com/paper/2410.03490/) (2024)
- [LLM Agents for Interactive Workflow Provenance: Reference Architecture and Evaluation Methodology](https://mlsanity.com/paper/2509.13978/) (2025)
- [AI Data Readiness Inspector (AIDRIN) for Quantitative Assessment of Data Readiness for AI](https://mlsanity.com/paper/2406.19256/) (2024)
- [Towards Lightweight Data Integration using Multi-workflow Provenance and Data Observability](https://mlsanity.com/paper/2308.09004/) (2023)
- [Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNN](https://mlsanity.com/paper/2406.12909/) (2024)
- [Towards a Foundation Model for Partial Differential Equations Across Physics Domains](https://mlsanity.com/paper/2511.21861/) (2025)
- [Do Large Language Models Speak Scientific Workflows?](https://mlsanity.com/paper/2412.10606/) (2024)
- [A Terminology for Scientific Workflow Systems](https://mlsanity.com/paper/2506.07838/) (2025)
- [MATEY: multiscale adaptive foundation models for spatiotemporal physical systems](https://mlsanity.com/paper/2412.20601/) (2024)

## Built on by (most cited first, 1 of 1)

- [SetGo: Metadata Readiness for Scientific AI Datasets](https://mlsanity.com/paper/2607.22677/) (2026)

## Abstract (arXiv)

Leadership computing facilities steward large-scale scientific datasets that routinely require substantial transformation before serving as AI training data. However, no existing framework fully unifies automated transformation, readiness assessment, provenance tracking, and agent-native deployment. We present REDI, an open-source framework that addresses this gap through a unified five-stage pipeline (ingest, preprocess, transform, structure, and output) with per-stage instrumentation for reproducibility and deployment as an agent-callable skill; companion tool SetGo automates FAIR compliance and catalog publication. Evaluated across climate, proteomics, materials science, and nuclear fusion, REDI transforms all datasets from raw to AI-ready, with outputs validated against domain-expert references, and preliminary results show near-ideal parallel scaling to 100 nodes on Frontier for the climate case. Provenance-instrumented profiling reveals file I/O as the dominant pipeline cost, with format selection a first-order optimization lever. These results establish REDI as a cross-domain platform providing automated data readiness for scientific AI, transforming data preparation bottlenecks into reproducible, reusable community assets.