# Holistic Interpretation of Public Scenes Using Computer Vision and Temporal Graphs to Identify Social Distancing Violations

arXiv: https://arxiv.org/abs/2112.06428 · HTML: https://mlsanity.com/paper/2112.06428/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 85 references that are in the map.
- First posted on arXiv 13 Dec 2021.

## Builds on (most cited first, 18 of 18)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking](https://mlsanity.com/paper/1504.01942/) (2015)
- [Fast YOLO: A Fast You Only Look Once System for Real-time Embedded Object Detection in Video](https://mlsanity.com/paper/1709.05943/) (2017)
- [Monitoring COVID-19 social distancing with person detection and tracking via fine-tuned YOLO v3 and Deepsort techniques](https://mlsanity.com/paper/2005.01385/) (2020)
- [A Vision-based Social Distancing and Critical Density Detection System for COVID-19](https://mlsanity.com/paper/2007.03578/) (2020)
- [DeepSOCIAL: Social Distancing Monitoring and Infection Risk Assessment in COVID-19 Pandemic](https://mlsanity.com/paper/2008.11672/) (2020)
- [A Tiny CNN Architecture for Medical Face Mask Detection for Resource-Constrained Endpoints](https://mlsanity.com/paper/2011.14858/) (2020)
- [Efficient Pedestrian Detection in Top-View Fisheye Images Using Compositions of Perspective View Patches](https://mlsanity.com/paper/2009.02711/) (2020)
- [Detecting Personality and Emotion Traits in Crowds from Video Sequences](https://mlsanity.com/paper/2104.12927/) (2021)
- [Potential Escalator-related Injury Identification and Prevention Based on Multi-module Integrated System for Public Health](https://mlsanity.com/paper/2103.07620/) (2021)
- [Hands Off: A Handshake Interaction Detection and Localization Model for COVID-19 Threat Control](https://mlsanity.com/paper/2110.09571/) (2021)

## Abstract (arXiv)

The COVID-19 pandemic has caused an unprecedented global public health crisis. Given its inherent nature, social distancing measures are proposed as the primary strategies to curb the spread of this pandemic. Therefore, identifying situations where these protocols are violated, has implications for curtailing the spread of the disease and promoting a sustainable lifestyle. This paper proposes a novel computer vision-based system to analyze CCTV footage to provide a threat level assessment of COVID-19 spread. The system strives to holistically capture and interpret the information content of CCTV footage spanning multiple frames to recognize instances of various violations of social distancing protocols, across time and space, as well as identification of group behaviors. This functionality is achieved primarily by utilizing a temporal graph-based structure to represent the information of the CCTV footage and a strategy to holistically interpret the graph and quantify the threat level of the given scene. The individual components are tested and validated on a range of scenarios and the complete system is tested against human expert opinion. The results reflect the dependence of the threat level on people, their physical proximity, interactions, protective clothing, and group dynamics. The system performance has an accuracy of 76%, thus enabling a deployable threat monitoring system in cities, to permit normalcy and sustainability in the society.