# NoScope: Optimizing Neural Network Queries over Video at Scale

arXiv: https://arxiv.org/abs/1703.02529 · HTML: https://mlsanity.com/paper/1703.02529/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2017.

## Built on by (most cited first, 30 of 37)

- [DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing](https://mlsanity.com/paper/2410.12189/) (2024)
- [Data-driven Discovery with Large Generative Models](https://mlsanity.com/paper/2402.13610/) (2024)
- [Abacus: A Cost-Based Optimizer for Semantic Operator Systems](https://mlsanity.com/paper/2505.14661/) (2025)
- [Shoggoth: Towards Efficient Edge-Cloud Collaborative Real-Time Video Inference via Adaptive Online Learning](https://mlsanity.com/paper/2306.15333/) (2023)
- [Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs](https://mlsanity.com/paper/2104.09455/) (2021)
- [Deep Neural Network-based Enhancement for Image and Video Streaming Systems: A Survey and Future Directions](https://mlsanity.com/paper/2106.03727/) (2021)
- [Edge-Based Video Analytics: A Survey](https://mlsanity.com/paper/2303.14329/) (2023)
- [AccMPEG: Optimizing Video Encoding for Video Analytics](https://mlsanity.com/paper/2204.12534/) (2022)
- [On Efficient Approximate Queries over Machine Learning Models](https://mlsanity.com/paper/2206.02845/) (2022)
- [Turbo: Opportunistic Enhancement for Edge Video Analytics](https://mlsanity.com/paper/2207.00172/) (2022)
- [Survey: Exploiting Data Redundancy for Optimization of Deep Learning](https://mlsanity.com/paper/2208.13363/) (2022)
- [A novel efficient Multi-view traffic-related object detection framework](https://mlsanity.com/paper/2302.11810/) (2023)
- [D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse](https://mlsanity.com/paper/2506.14107/) (2025)
- [Enhancing Video Analytics Accuracy via Real-time Automated Camera Parameter Tuning](https://mlsanity.com/paper/2107.03964/) (2021)
- [FrameHopper: Selective Processing of Video Frames in Detection-driven Real-Time Video Analytics](https://mlsanity.com/paper/2203.11493/) (2022)
- [Zeus: Efficiently Localizing Actions in Videos using Reinforcement Learning](https://mlsanity.com/paper/2104.06142/) (2021)
- [Why is the video analytics accuracy fluctuating, and what can we do about it?](https://mlsanity.com/paper/2208.12644/) (2022)
- [ScaleDoc: Scaling LLM-based Predicates over Large Document Collections](https://mlsanity.com/paper/2509.12610/) (2025)
- [Towards Automatic Model Specialization for Edge Video Analytics](https://mlsanity.com/paper/2104.06826/) (2021)
- [OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation](https://mlsanity.com/paper/2310.02422/) (2023)
- [Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines](https://mlsanity.com/paper/2405.11191/) (2024)
- [Empower Vision Applications with LoRA LMM](https://mlsanity.com/paper/2411.00915/) (2024)
- [Croesus: Multi-Stage Processing and Transactions for Video-Analytics in Edge-Cloud Systems](https://mlsanity.com/paper/2201.00063/) (2021)
- [Spatialyze: A Geospatial Video Analytics System with Spatial-Aware Optimizations](https://mlsanity.com/paper/2308.03276/) (2023)
- [BiSwift: Bandwidth Orchestrator for Multi-Stream Video Analytics on Edge](https://mlsanity.com/paper/2312.15740/) (2023)
- [Personalized Top-k Set Queries Over Predicted Scores](https://mlsanity.com/paper/2502.12998/) (2025)
- [HybridServe: Efficient Serving of Large AI Models with Confidence-Based Cascade Routing](https://mlsanity.com/paper/2505.12566/) (2025)
- [A Survey of Performance Optimization in Neural Network-Based Video Analytics Systems](https://mlsanity.com/paper/2105.14195/) (2021)
- [Elixir: A system to enhance data quality for multiple analytics on a video stream](https://mlsanity.com/paper/2212.04061/) (2022)
- [Adaptive Scheduling for Edge-Assisted DNN Serving](https://mlsanity.com/paper/2304.09961/) (2023)

## Abstract (arXiv)

Recent advances in computer vision-in the form of deep neural networks-have made it possible to query increasing volumes of video data with high accuracy. However, neural network inference is computationally expensive at scale: applying a state-of-the-art object detector in real time (i.e., 30+ frames per second) to a single video requires a 4000 GPU. In response, we present NoScope, a system for querying videos that can reduce the cost of neural network video analysis by up to three orders of magnitude via inference-optimized model search. Given a target video, object to detect, and reference neural network, NoScope automatically searches for and trains a sequence, or cascade, of models that preserves the accuracy of the reference network but is specialized to the target video and are therefore far less computationally expensive. NoScope cascades two types of models: specialized models that forego the full generality of the reference model but faithfully mimic its behavior for the target video and object; and difference detectors that highlight temporal differences across frames. We show that the optimal cascade architecture differs across videos and objects, so NoScope uses an efficient cost-based optimizer to search across models and cascades. With this approach, NoScope achieves two to three order of magnitude speed-ups (265-15,500x real-time) on binary classification tasks over fixed-angle webcam and surveillance video while maintaining accuracy within 1-5% of state-of-the-art neural networks.