# Extract-Transform-Load for Video Streams

arXiv: https://arxiv.org/abs/2310.04830 · HTML: https://mlsanity.com/paper/2310.04830/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 93 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [Large scale distributed neural network training through online distillation](https://mlsanity.com/paper/1804.03235/) (2018)
- [TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking](https://mlsanity.com/paper/2104.00194/) (2021)
- [Device Placement Optimization with Reinforcement Learning](https://mlsanity.com/paper/1706.04972/) (2017)
- [Focus: Querying Large Video Datasets with Low Latency and Low Cost](https://mlsanity.com/paper/1801.03493/) (2018)
- [BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics](https://mlsanity.com/paper/1805.01046/) (2018)
- [MaskedFace-Net -- A Dataset of Correctly/Incorrectly Masked Face Images in the Context of COVID-19](https://mlsanity.com/paper/2008.08016/) (2020)
- [Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning](https://mlsanity.com/paper/1906.08879/) (2019)
- [Physical Representation-based Predicate Optimization for a Visual Analytics Database](https://mlsanity.com/paper/1806.04226/) (2018)
- [Llama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines](https://mlsanity.com/paper/2102.01887/) (2021)
- [Optimizing Machine Learning Inference Queries with Correlative Proxy Models](https://mlsanity.com/paper/2201.00309/) (2022)
- [Semantic Indexes for Machine Learning-based Queries over Unstructured Data](https://mlsanity.com/paper/2009.04540/) (2020)
- [Accelerating Approximate Aggregation Queries with Expensive Predicates](https://mlsanity.com/paper/2108.06313/) (2021)
- [VStore: A Data Store for Analytics on Large Videos](https://mlsanity.com/paper/1810.01794/) (2018)
- [THIA: Accelerating Video Analytics using Early Inference and Fine-Grained Query Planning](https://mlsanity.com/paper/2102.08481/) (2021)
- [TASM: A Tile-Based Storage Manager for Video Analytics](https://mlsanity.com/paper/2006.02958/) (2020)
- [Evolving Boxes for Fast Vehicle Detection](https://mlsanity.com/paper/1702.00254/) (2017)
- [Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics](https://mlsanity.com/paper/2001.00459/) (2019)
- [Detection of Glottal Closure Instants from Speech Signals: a Quantitative Review](https://mlsanity.com/paper/2001.00473/) (2019)

## Built on by (most cited first, 7 of 7)

- [Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook](https://mlsanity.com/paper/2503.18016/) (2025)
- [iRAG: Advancing RAG for Videos with an Incremental Approach](https://mlsanity.com/paper/2404.12309/) (2024)
- [Abacus: A Cost-Based Optimizer for Semantic Operator Systems](https://mlsanity.com/paper/2505.14661/) (2025)
- [CascadeServe: Unlocking Model Cascades for Inference Serving](https://mlsanity.com/paper/2406.14424/) (2024)
- [Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMs](https://mlsanity.com/paper/2410.17840/) (2024)
- [TrafficLens: Multi-Camera Traffic Video Analysis Using LLMs](https://mlsanity.com/paper/2511.20965/) (2025)
- [Energy-Efficient Autonomous Driving with Adaptive Perception and Robust Decision](https://mlsanity.com/paper/2510.25205/) (2025)

## Abstract (arXiv)

Social media, self-driving cars, and traffic cameras produce video streams at large scales and cheap cost. However, storing and querying video at such scales is prohibitively expensive. We propose to treat large-scale video analytics as a data warehousing problem: Video is a format that is easy to produce but needs to be transformed into an application-specific format that is easy to query. Analogously, we define the problem of Video Extract-Transform-Load (V-ETL). V-ETL systems need to reduce the cost of running a user-defined V-ETL job while also giving throughput guarantees to keep up with the rate at which data is produced. We find that no current system sufficiently fulfills both needs and therefore propose Skyscraper, a system tailored to V-ETL. Skyscraper can execute arbitrary video ingestion pipelines and adaptively tunes them to reduce cost at minimal or no quality degradation, e.g., by adjusting sampling rates and resolutions to the ingested content. Skyscraper can hereby be provisioned with cheap on-premises compute and uses a combination of buffering and cloud bursting to deal with peaks in workload caused by expensive processing configurations. In our experiments, we find that Skyscraper significantly reduces the cost of V-ETL ingestion compared to adaptions of current SOTA systems, while at the same time giving robustness guarantees that these systems are lacking.