# Optimizing Machine Learning Inference Queries with Correlative Proxy Models

arXiv: https://arxiv.org/abs/2201.00309 · HTML: https://mlsanity.com/paper/2201.00309/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jan 2022.

## Built on by (most cited first, 13 of 13)

- [Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures](https://mlsanity.com/paper/2205.04713/) (2022)
- [On Efficient Approximate Queries over Machine Learning Models](https://mlsanity.com/paper/2206.02845/) (2022)
- [Extract-Transform-Load for Video Streams](https://mlsanity.com/paper/2310.04830/) (2023)
- [Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving](https://mlsanity.com/paper/2504.07494/) (2025)
- [Batch Query Processing and Optimization for Agentic Workflows](https://mlsanity.com/paper/2509.02121/) (2025)
- [Powering In-Database Dynamic Model Slicing for Structured Data Analytics](https://mlsanity.com/paper/2405.00568/) (2024)
- [ScaleDoc: Scaling LLM-based Predicates over Large Document Collections](https://mlsanity.com/paper/2509.12610/) (2025)
- [The Duck's Brain: Training and Inference of Neural Networks in Modern Database Engines](https://mlsanity.com/paper/2312.17355/) (2023)
- [VQPy: An Object-Oriented Approach to Modern Video Analytics](https://mlsanity.com/paper/2311.01623/) (2023)
- [Personalized Top-k Set Queries Over Predicted Scores](https://mlsanity.com/paper/2502.12998/) (2025)
- [Beyond Linear LLM Invocation: An Efficient and Effective Semantic Filter Paradigm](https://mlsanity.com/paper/2603.04799/) (2026)
- [Hydro: Adaptive Query Processing of ML Queries](https://mlsanity.com/paper/2403.14902/) (2024)
- [Artificial Intelligence for Green Hydrogen Yield Prediction and Site Suitability using SHAP-Based Composite Index: Focus on Oman](https://mlsanity.com/paper/2507.14219/) (2025)

## Abstract (arXiv)

We consider accelerating machine learning (ML) inference queries on unstructured datasets. Expensive operators such as feature extractors and classifiers are deployed as user-defined functions(UDFs), which are not penetrable with classic query optimization techniques such as predicate push-down. Recent optimization schemes (e.g., Probabilistic Predicates or PP) assume independence among the query predicates, build a proxy model for each predicate offline, and rewrite a new query by injecting these cheap proxy models in the front of the expensive ML UDFs. In such a manner, unlikely inputs that do not satisfy query predicates are filtered early to bypass the ML UDFs. We show that enforcing the independence assumption in this context may result in sub-optimal plans. In this paper, we propose CORE, a query optimizer that better exploits the predicate correlations and accelerates ML inference queries. Our solution builds the proxy models online for a new query and leverages a branch-and-bound search process to reduce the building costs. Results on three real-world text, image and video datasets show that CORE improves the query throughput by up to 63% compared to PP and up to 80% compared to running the queries as it is.