# Prompt2DAG: A Modular Methodology for LLM-Based Data Enrichment Pipeline Generation

arXiv: https://arxiv.org/abs/2509.13487 · HTML: https://mlsanity.com/paper/2509.13487/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2025.

## Built on by (most cited first, 3 of 3)

- [DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving](https://mlsanity.com/paper/2606.07001/) (2026)
- [Autonomous Data Processing using Meta-Agents](https://mlsanity.com/paper/2602.00307/) (2026)
- [Tool-Aware Planning in Contact Center AI: Evaluating LLMs through Lineage-Guided Query Decomposition](https://mlsanity.com/paper/2602.14955/) (2026)

## Abstract (arXiv)

Developing reliable data enrichment pipelines demands significant engineering expertise. We present Prompt2DAG, a methodology that transforms natural language descriptions into executable Apache Airflow DAGs. We evaluate four generation approaches -- Direct, LLM-only, Hybrid, and Template-based -- across 260 experiments using thirteen LLMs and five case studies to identify optimal strategies for production-grade automation. Performance is measured using a penalized scoring framework that combines reliability with code quality (SAT), structural integrity (DST), and executability (PCT). The Hybrid approach emerges as the optimal generative method, achieving a 78.5% success rate with robust quality scores (SAT: 6.79, DST: 7.67, PCT: 7.76). This significantly outperforms the LLM-only (66.2% success) and Direct (29.2% success) methods. Our findings show that reliability, not intrinsic code quality, is the primary differentiator. Cost-effectiveness analysis reveals the Hybrid method is over twice as efficient as Direct prompting per successful DAG. We conclude that a structured, hybrid approach is essential for balancing flexibility and reliability in automated workflow generation, offering a viable path to democratize data pipeline development.