# Video Monitoring Queries

arXiv: https://arxiv.org/abs/2002.10537 · HTML: https://mlsanity.com/paper/2002.10537/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2020.

## Built on by (most cited first, 7 of 7)

- [CoVA: Exploiting Compressed-Domain Analysis to Accelerate Video Analytics](https://mlsanity.com/paper/2207.00588/) (2022)
- [ScanBank: A Benchmark Dataset for Figure Extraction from Scanned Electronic Theses and Dissertations](https://mlsanity.com/paper/2106.15320/) (2021)
- [Pool of Experts: Realtime Querying Specialized Knowledge in Massive Neural Networks](https://mlsanity.com/paper/2107.01354/) (2021)
- [Automatic and Efficient Customization of Neural Networks for ML Applications](https://mlsanity.com/paper/2310.04685/) (2023)
- [Hydro: Adaptive Query Processing of ML Queries](https://mlsanity.com/paper/2403.14902/) (2024)
- [TRACER: Efficient Object Re-Identification in Networked Cameras through Adaptive Query Processing](https://mlsanity.com/paper/2507.09448/) (2025)
- [LAVA: Language Driven Scalable and Versatile Traffic Video Analytics](https://mlsanity.com/paper/2507.19821/) (2025)

## Abstract (arXiv)

Recent advances in video processing utilizing deep learning primitives achieved breakthroughs in fundamental problems in video analysis such as frame classification and object detection enabling an array of new applications. In this paper we study the problem of interactive declarative query processing on video streams. In particular we introduce a set of approximate filters to speed up queries that involve objects of specific type (e.g., cars, trucks, etc.) on video frames with associated spatial relationships among them (e.g., car left of truck). The resulting filters are able to assess quickly if the query predicates are true to proceed with further analysis of the frame or otherwise not consider the frame further avoiding costly object detection operations. We propose two classes of filters IC and OD, that adapt principles from deep image classification and object detection. The filters utilize extensible deep neural architectures and are easy to deploy and utilize. In addition, we propose statistical query processing techniques to process aggregate queries involving objects with spatial constraints on video streams and demonstrate experimentally the resulting increased accuracy on the resulting aggregate estimation. Combined these techniques constitute a robust set of video monitoring query processing techniques. We demonstrate that the application of the techniques proposed in conjunction with declarative queries on video streams can dramatically increase the frame processing rate and speed up query processing by at least two orders of magnitude. We present the results of a thorough experimental study utilizing benchmark video data sets at scale demonstrating the performance benefits and the practical relevance of our proposals.