# Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics

arXiv: https://arxiv.org/abs/2106.15315 · HTML: https://mlsanity.com/paper/2106.15315/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 147 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 30 of 45)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Trained Ternary Quantization](https://mlsanity.com/paper/1612.01064/) (2016)
- [Speed/accuracy trade-offs for modern convolutional object detectors](https://mlsanity.com/paper/1611.10012/) (2016)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Memory Enhanced Global-Local Aggregation for Video Object Detection](https://mlsanity.com/paper/2003.12063/) (2020)
- [Relation Distillation Networks for Video Object Detection](https://mlsanity.com/paper/1908.09511/) (2019)
- [T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos](https://mlsanity.com/paper/1604.02532/) (2016)
- [Repulsion Loss: Detecting Pedestrians in a Crowd](https://mlsanity.com/paper/1711.07752/) (2017)
- [Towards High Performance Video Object Detection](https://mlsanity.com/paper/1711.11577/) (2017)
- [Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers](https://mlsanity.com/paper/2012.10557/) (2020)
- [DeepCache: Principled Cache for Mobile Deep Vision](https://mlsanity.com/paper/1712.01670/) (2017)
- [Object Detection in Video with Spatiotemporal Sampling Networks](https://mlsanity.com/paper/1803.05549/) (2018)
- [Focus: Querying Large Video Datasets with Low Latency and Low Cost](https://mlsanity.com/paper/1801.03493/) (2018)
- [BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics](https://mlsanity.com/paper/1805.01046/) (2018)
- [Object Detection from Video Tubelets with Convolutional Neural Networks](https://mlsanity.com/paper/1604.04053/) (2016)
- [Scaling Video Analytics on Constrained Edge Nodes](https://mlsanity.com/paper/1905.13536/) (2019)
- [Fast Object Detection in Compressed Video](https://mlsanity.com/paper/1811.11057/) (2018)

## Built on by (most cited first, 10 of 10)

- [Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving](https://mlsanity.com/paper/2312.05385/) (2023)
- [Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques](https://mlsanity.com/paper/2211.15751/) (2022)
- [MadEye: Boosting Live Video Analytics Accuracy with Adaptive Camera Configurations](https://mlsanity.com/paper/2304.02101/) (2023)
- [D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse](https://mlsanity.com/paper/2506.14107/) (2025)
- [CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference](https://mlsanity.com/paper/2604.06036/) (2026)
- [OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation](https://mlsanity.com/paper/2310.02422/) (2023)
- [Legilimens: Performant Video Analytics on the System-on-Chip Edge](https://mlsanity.com/paper/2504.21136/) (2025)
- [T-TAMER: Provably Taming Trade-offs in ML Serving](https://mlsanity.com/paper/2509.22992/) (2025)
- [RedunCut: Measurement-Driven Sampling and Accuracy Performance Modeling for Low-Cost Live Video Analytics](https://mlsanity.com/paper/2512.24386/) (2025)
- [Search over the Visual World: Persistent Visual Memory, Layered Indexes, and Source-Grounded Evidence](https://mlsanity.com/paper/2608.08075/) (2026)

## Abstract (arXiv)

Commercial retrospective video analytics platforms have increasingly adopted general interfaces to support the custom queries and convolutional neural networks (CNNs) that different applications require. However, existing optimizations were designed for settings where CNNs were platform- (not user-) determined, and fail to meet at least one of the following key platform goals when that condition is violated: reliable accuracy, low latency, and minimal wasted work. We present Boggart, a system that simultaneously meets all three goals while supporting the generality that today's platforms seek. Prior to queries being issued, Boggart carefully employs traditional computer vision algorithms to generate indices that are imprecise, but are fundamentally comprehensive across different CNNs/queries. For each issued query, Boggart employs new techniques to quickly characterize the imprecision of its index, and sparingly run CNNs (and propagate the results to other frames) in a way that bounds accuracy drops. Our results highlight that Boggart's improved generality comes at low cost, with speedups that match (and most often, exceed) prior, model-specific approaches.