# End-to-end Optimization of Machine Learning Prediction Queries

arXiv: https://arxiv.org/abs/2206.00136 · HTML: https://mlsanity.com/paper/2206.00136/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 79 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [OpenML Benchmarking Suites](https://mlsanity.com/paper/1708.03731/) (2017)
- [MLlib: Machine Learning in Apache Spark](https://mlsanity.com/paper/1505.06807/) (2015)
- [tf.data: A Machine Learning Data Processing Framework](https://mlsanity.com/paper/2101.12127/) (2021)
- [Rafiki: Machine Learning as an Analytics Service System](https://mlsanity.com/paper/1804.06087/) (2018)
- [PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems](https://mlsanity.com/paper/1810.06115/) (2018)
- [A Tensor Compiler for Unified Machine Learning Prediction Serving](https://mlsanity.com/paper/2010.04804/) (2020)
- [Extending Relational Query Processing with ML Inference](https://mlsanity.com/paper/1911.00231/) (2019)
- [KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analytics](https://mlsanity.com/paper/1610.09451/) (2016)
- [Machine Learning at Microsoft with ML .NET](https://mlsanity.com/paper/1905.05715/) (2019)
- [Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics](https://mlsanity.com/paper/2007.13005/) (2020)
- [sql4ml A declarative end-to-end workflow for machine learning](https://mlsanity.com/paper/1907.12415/) (2019)
- [Cloudy with high chance of DBMS: A 10-year prediction for Enterprise-Grade ML](https://mlsanity.com/paper/1909.00084/) (2019)
- [The Relational Data Borg is Learning](https://mlsanity.com/paper/2008.07864/) (2020)
- [LaraDB: A Minimalist Kernel for Linear and Relational Algebra Computation](https://mlsanity.com/paper/1703.07342/) (2017)
- [Optimization of Imperative Programs in a Relational Database](https://mlsanity.com/paper/1712.00498/) (2017)
- [Data Science through the looking glass and what we found there](https://mlsanity.com/paper/1912.09536/) (2019)
- [SPORES: Sum-Product Optimization via Relational Equality Saturation for Large Scale Linear Algebra](https://mlsanity.com/paper/2002.07951/) (2020)
- [MLI: An API for Distributed Machine Learning](https://mlsanity.com/paper/1310.5426/) (2013)
- [Willump: A Statistically-Aware End-to-end Optimizer for Machine Learning Inference](https://mlsanity.com/paper/1906.01974/) (2019)

## Built on by (most cited first, 16 of 16)

- [GFS: Graph-based Feature Synthesis for Prediction over Relational Databases](https://mlsanity.com/paper/2312.02037/) (2023)
- [Predictive Query Language: A Domain-Specific Language for Predictive Modeling on Relational Databases](https://mlsanity.com/paper/2602.09572/) (2026)
- [Share the Tensor Tea: How Databases can Leverage the Machine Learning Ecosystem](https://mlsanity.com/paper/2209.04579/) (2022)
- [Powering In-Database Dynamic Model Slicing for Structured Data Analytics](https://mlsanity.com/paper/2405.00568/) (2024)
- [NeurDB: An AI-powered Autonomous Data System](https://mlsanity.com/paper/2405.03924/) (2024)
- [NeurDB: On the Design and Implementation of an AI-powered Autonomous Database](https://mlsanity.com/paper/2408.03013/) (2024)
- [The Tensor Data Platform: Towards an AI-centric Database System](https://mlsanity.com/paper/2211.02753/) (2022)
- [CityGPT: Towards Urban IoT Learning, Analysis and Interaction with Multi-Agent System](https://mlsanity.com/paper/2405.14691/) (2024)
- [Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines](https://mlsanity.com/paper/2405.11191/) (2024)
- [NeurStore: Efficient In-database Deep Learning Model Management System](https://mlsanity.com/paper/2509.03228/) (2025)
- [Auto-Differentiation of Relational Computations for Very Large Scale Machine Learning](https://mlsanity.com/paper/2306.00088/) (2023)
- [Bullion: A Column Store for Machine Learning](https://mlsanity.com/paper/2404.08901/) (2024)
- [Beyond Linear LLM Invocation: An Efficient and Effective Semantic Filter Paradigm](https://mlsanity.com/paper/2603.04799/) (2026)
- [Keep Your Friends Close: Leveraging Affinity Groups to Accelerate AI Inference Workflows](https://mlsanity.com/paper/2312.11488/) (2023)
- [InferF: Declarative Factorization of AI/ML Inferences over Joins](https://mlsanity.com/paper/2511.20489/) (2025)
- [Towards Effective Orchestration of AI x DB Workloads](https://mlsanity.com/paper/2603.03772/) (2026)

## Abstract (arXiv)

Prediction queries are widely used across industries to perform advanced analytics and draw insights from data. They include a data processing part (e.g., for joining, filtering, cleaning, featurizing the datasets) and a machine learning (ML) part invoking one or more trained models to perform predictions. These parts have so far been optimized in isolation, leaving significant opportunities for optimization unexplored. We present Raven, a production-ready system for optimizing prediction queries. Raven follows the enterprise architectural trend of collocating data and ML runtimes. It relies on a unified intermediate representation that captures both data and ML operators in a single graph structure to unlock two families of optimizations. First, it employs logical optimizations that pass information between the data part (and the properties of the underlying data) and the ML part to optimize each other. Second, it introduces logical-to-physical transformations that allow operators to be executed on different runtimes (relational, ML, and DNN) and hardware (CPU, GPU). Novel data-driven optimizations determine the runtime to be used for each part of the query to achieve optimal performance. Our evaluation shows that Raven improves performance of prediction queries on Apache Spark and SQL Server by up to 13.1x and 330x, respectively. For complex models where GPU acceleration is beneficial, Raven provides up to 8x speedup compared to state-of-the-art systems.