# BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics

arXiv: https://arxiv.org/abs/1805.01046 · HTML: https://mlsanity.com/paper/1805.01046/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2018.

## Built on by (most cited first, 30 of 41)

- [Semantic Operators: A Declarative Model for Rich, AI-based Data Processing](https://mlsanity.com/paper/2407.11418/) (2024)
- [Abacus: A Cost-Based Optimizer for Semantic Operator Systems](https://mlsanity.com/paper/2505.14661/) (2025)
- [Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures](https://mlsanity.com/paper/2205.04713/) (2022)
- [Cortex AISQL: A Production SQL Engine for Unstructured Data](https://mlsanity.com/paper/2511.07663/) (2025)
- [Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics](https://mlsanity.com/paper/2106.15315/) (2021)
- [Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs](https://mlsanity.com/paper/2104.09455/) (2021)
- [Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees](https://mlsanity.com/paper/2509.02896/) (2025)
- [AccMPEG: Optimizing Video Encoding for Video Analytics](https://mlsanity.com/paper/2204.12534/) (2022)
- [Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques](https://mlsanity.com/paper/2211.15751/) (2022)
- [MadEye: Boosting Live Video Analytics Accuracy with Adaptive Camera Configurations](https://mlsanity.com/paper/2304.02101/) (2023)
- [Extract-Transform-Load for Video Streams](https://mlsanity.com/paper/2310.04830/) (2023)
- [Multi-Modal Data Exploration via Language Agents](https://mlsanity.com/paper/2412.18428/) (2024)
- [Deep Research is the New Analytics System: Towards Building the Runtime for AI-Driven Analytics](https://mlsanity.com/paper/2509.02751/) (2025)
- [CoVA: Exploiting Compressed-Domain Analysis to Accelerate Video Analytics](https://mlsanity.com/paper/2207.00588/) (2022)
- [Batch Query Processing and Optimization for Agentic Workflows](https://mlsanity.com/paper/2509.02121/) (2025)
- [D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse](https://mlsanity.com/paper/2506.14107/) (2025)
- [Zeus: Efficiently Localizing Actions in Videos using Reinforcement Learning](https://mlsanity.com/paper/2104.06142/) (2021)
- [Towards Automatic Model Specialization for Edge Video Analytics](https://mlsanity.com/paper/2104.06826/) (2021)
- [VOCALExplore: Pay-as-You-Go Video Data Exploration and Model Building [Technical Report]](https://mlsanity.com/paper/2303.04068/) (2023)
- [OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation](https://mlsanity.com/paper/2310.02422/) (2023)
- [Empower Vision Applications with LoRA LMM](https://mlsanity.com/paper/2411.00915/) (2024)
- [Pool of Experts: Realtime Querying Specialized Knowledge in Massive Neural Networks](https://mlsanity.com/paper/2107.01354/) (2021)
- [VQPy: An Object-Oriented Approach to Modern Video Analytics](https://mlsanity.com/paper/2311.01623/) (2023)
- [SketchQL Demonstration: Zero-shot Video Moment Querying with Sketches](https://mlsanity.com/paper/2405.18334/) (2024)
- [LazyVLM: Neuro-Symbolic Approach to Video Analytics](https://mlsanity.com/paper/2505.21459/) (2025)
- [Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet](https://mlsanity.com/paper/2507.19209/) (2025)
- [Beyond Linear LLM Invocation: An Efficient and Effective Semantic Filter Paradigm](https://mlsanity.com/paper/2603.04799/) (2026)
- [VID-WIN: Fast Video Event Matching with Query-Aware Windowing at the Edge for the Internet of Multimedia Things](https://mlsanity.com/paper/2105.02957/) (2021)
- [A Survey of Performance Optimization in Neural Network-Based Video Analytics Systems](https://mlsanity.com/paper/2105.14195/) (2021)
- [InFi: End-to-End Learning to Filter Input for Resource-Efficiency in Mobile-Centric Inference](https://mlsanity.com/paper/2209.13873/) (2022)

## Abstract (arXiv)

Recent advances in neural networks (NNs) have enabled automatic querying of large volumes of video data with high accuracy. While these deep NNs can produce accurate annotations of an object's position and type in video, they are computationally expensive and require complex, imperative deployment code to answer queries. Prior work uses approximate filtering to reduce the cost of video analytics, but does not handle two important classes of queries, aggregation and limit queries; moreover, these approaches still require complex code to deploy. To address the computational and usability challenges of querying video at scale, we introduce BlazeIt, a system that optimizes queries of spatiotemporal information of objects in video. BlazeIt accepts queries via FrameQL, a declarative extension of SQL for video analytics that enables video-specific query optimization. We introduce two new query optimization techniques in BlazeIt that are not supported by prior work. First, we develop methods of using NNs as control variates to quickly answer approximate aggregation queries with error bounds. Second, we present a novel search algorithm for cardinality-limited video queries. Through these these optimizations, BlazeIt can deliver up to 83x speedups over the recent literature on video processing.