# Zero-Shot Cost Models for Distributed Stream Processing

arXiv: https://arxiv.org/abs/2207.03823 · HTML: https://mlsanity.com/paper/2207.03823/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2022.

## Built on by (most cited first, 4 of 4)

- [PRICE: A Pretrained Model for Cross-Database Cardinality Estimation](https://mlsanity.com/paper/2406.01027/) (2024)
- [The Unreasonable Effectiveness of LLMs for Query Optimization](https://mlsanity.com/paper/2411.02862/) (2024)
- [The Duck's Brain: Training and Inference of Neural Networks in Modern Database Engines](https://mlsanity.com/paper/2312.17355/) (2023)
- [COSTREAM: Learned Cost Models for Operator Placement in Edge-Cloud Environments](https://mlsanity.com/paper/2403.08444/) (2024)

## Abstract (arXiv)

This paper proposes a learned cost estimation model for Distributed Stream Processing Systems (DSPS) with an aim to provide accurate cost predictions of executing queries. A major premise of this work is that the proposed learned model can generalize to the dynamics of streaming workloads out-of-the-box. This means a model once trained can accurately predict performance metrics such as latency and throughput even if the characteristics of the data and workload or the deployment of operators to hardware changes at runtime. That way, the model can be used to solve tasks such as optimizing the placement of operators to minimize the end-to-end latency of a streaming query or maximize its throughput even under varying conditions. Our evaluation on a well-known DSPS, Apache Storm, shows that the model can predict accurately for unseen workloads and queries while generalizing across real-world benchmarks.