# Towards Causal Physical Error Discovery in Video Analytics Systems

arXiv: https://arxiv.org/abs/2405.17686 · HTML: https://mlsanity.com/paper/2405.17686/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 12 of 12)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Focus: Querying Large Video Datasets with Low Latency and Low Cost](https://mlsanity.com/paper/1801.03493/) (2018)
- [BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics](https://mlsanity.com/paper/1805.01046/) (2018)
- [NoScope: Optimizing Neural Network Queries over Video at Scale](https://mlsanity.com/paper/1703.02529/) (2017)
- [MacroBase: Prioritizing Attention in Fast Data](https://mlsanity.com/paper/1603.00567/) (2016)
- [Rekall: Specifying Video Events using Compositions of Spatiotemporal Labels](https://mlsanity.com/paper/1910.02993/) (2019)
- [VStore: A Data Store for Analytics on Large Videos](https://mlsanity.com/paper/1810.01794/) (2018)
- [TASM: A Tile-Based Storage Manager for Video Analytics](https://mlsanity.com/paper/2006.02958/) (2020)
- [Scanner: Efficient Video Analysis at Scale](https://mlsanity.com/paper/1805.07339/) (2018)

## Abstract (arXiv)

Video analytics systems based on deep learning models are often opaque and brittle and require explanation systems to help users debug. Current model explanation system are very good at giving literal explanations of behavior in terms of pixel contributions but cannot integrate information about the physical or systems processes that might influence a prediction. This paper introduces the idea that a simple form of causal reasoning, called a regression discontinuity design, can be used to associate changes in multiple key performance indicators to physical real world phenomena to give users a more actionable set of video analytics explanations. We overview the system architecture and describe a vision of the impact that such a system might have.