# Physical Representation-based Predicate Optimization for a Visual Analytics Database

arXiv: https://arxiv.org/abs/1806.04226 · HTML: https://mlsanity.com/paper/1806.04226/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2018.

## Built on by (most cited first, 19 of 19)

- [A Declarative System for Optimizing AI Workloads](https://mlsanity.com/paper/2405.14696/) (2024)
- [Abacus: A Cost-Based Optimizer for Semantic Operator Systems](https://mlsanity.com/paper/2505.14661/) (2025)
- [Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics](https://mlsanity.com/paper/2106.15315/) (2021)
- [Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees](https://mlsanity.com/paper/2509.02896/) (2025)
- [On Efficient Approximate Queries over Machine Learning Models](https://mlsanity.com/paper/2206.02845/) (2022)
- [Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques](https://mlsanity.com/paper/2211.15751/) (2022)
- [Extract-Transform-Load for Video Streams](https://mlsanity.com/paper/2310.04830/) (2023)
- [Deep Research is the New Analytics System: Towards Building the Runtime for AI-Driven Analytics](https://mlsanity.com/paper/2509.02751/) (2025)
- [CoVA: Exploiting Compressed-Domain Analysis to Accelerate Video Analytics](https://mlsanity.com/paper/2207.00588/) (2022)
- [VOCALExplore: Pay-as-You-Go Video Data Exploration and Model Building [Technical Report]](https://mlsanity.com/paper/2303.04068/) (2023)
- [VQPy: An Object-Oriented Approach to Modern Video Analytics](https://mlsanity.com/paper/2311.01623/) (2023)
- [Personalized Top-k Set Queries Over Predicted Scores](https://mlsanity.com/paper/2502.12998/) (2025)
- [HybridServe: Efficient Serving of Large AI Models with Confidence-Based Cascade Routing](https://mlsanity.com/paper/2505.12566/) (2025)
- [VID-WIN: Fast Video Event Matching with Query-Aware Windowing at the Edge for the Internet of Multimedia Things](https://mlsanity.com/paper/2105.02957/) (2021)
- [A Survey of Performance Optimization in Neural Network-Based Video Analytics Systems](https://mlsanity.com/paper/2105.14195/) (2021)
- [MaskSearch: Querying Image Masks at Scale](https://mlsanity.com/paper/2305.02375/) (2023)
- [Automatic and Efficient Customization of Neural Networks for ML Applications](https://mlsanity.com/paper/2310.04685/) (2023)
- [Hydro: Adaptive Query Processing of ML Queries](https://mlsanity.com/paper/2403.14902/) (2024)
- [RedunCut: Measurement-Driven Sampling and Accuracy Performance Modeling for Low-Cost Live Video Analytics](https://mlsanity.com/paper/2512.24386/) (2025)

## Abstract (arXiv)

Querying the content of images, video, and other non-textual data sources requires expensive content extraction methods. Modern extraction techniques are based on deep convolutional neural networks (CNNs) and can classify objects within images with astounding accuracy. Unfortunately, these methods are slow: processing a single image can take about 10 milliseconds on modern GPU-based hardware. As massive video libraries become ubiquitous, running a content-based query over millions of video frames is prohibitive. One promising approach to reduce the runtime cost of queries of visual content is to use a hierarchical model, such as a cascade, where simple cases are handled by an inexpensive classifier. Prior work has sought to design cascades that optimize the computational cost of inference by, for example, using smaller CNNs. However, we observe that there are critical factors besides the inference time that dramatically impact the overall query time. Notably, by treating the physical representation of the input image as part of our query optimization---that is, by including image transforms, such as resolution scaling or color-depth reduction, within the cascade---we can optimize data handling costs and enable drastically more efficient classifier cascades. In this paper, we propose Tahoma, which generates and evaluates many potential classifier cascades that jointly optimize the CNN architecture and input data representation. Our experiments on a subset of ImageNet show that Tahoma's input transformations speed up cascades by up to 35 times. We also find up to a 98x speedup over the ResNet50 classifier with no loss in accuracy, and a 280x speedup if some accuracy is sacrificed.