# Covy: An AI-powered Robot with a Compound Vision System for Detecting Breaches in Social Distancing

arXiv: https://arxiv.org/abs/2207.06847 · HTML: https://mlsanity.com/paper/2207.06847/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 14 Jul 2022.

## Builds on (most cited first, 24 of 24)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [PifPaf: Composite Fields for Human Pose Estimation](https://mlsanity.com/paper/1903.06593/) (2019)
- [Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation](https://mlsanity.com/paper/1703.00420/) (2017)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation](https://mlsanity.com/paper/1709.10489/) (2017)
- [MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation](https://mlsanity.com/paper/1906.06059/) (2019)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [F-formation Detection: Individuating Free-standing Conversational Groups in Images](https://mlsanity.com/paper/1409.2702/) (2014)
- [Monitoring COVID-19 social distancing with person detection and tracking via fine-tuned YOLO v3 and Deepsort techniques](https://mlsanity.com/paper/2005.01385/) (2020)
- [Deep Reinforcement learning for real autonomous mobile robot navigation in indoor environments](https://mlsanity.com/paper/2005.13857/) (2020)
- [Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning](https://mlsanity.com/paper/2010.10903/) (2020)
- [Vision-based Navigation Using Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03627/) (2019)
- [DeepSOCIAL: Social Distancing Monitoring and Infection Risk Assessment in COVID-19 Pandemic](https://mlsanity.com/paper/2008.11672/) (2020)
- [Perceiving Humans: from Monocular 3D Localization to Social Distancing](https://mlsanity.com/paper/2009.00984/) (2020)
- [COVID-Robot: Monitoring Social Distancing Constraints in Crowded Scenarios](https://mlsanity.com/paper/2008.06585/) (2020)
- [Autonomous Social Distancing in Urban Environments using a Quadruped Robot](https://mlsanity.com/paper/2008.08889/) (2020)

## Abstract (arXiv)

This paper introduces a compound vision system that enables robots to localize people up to 15m away using a cheap camera. And, it proposes a robust navigation stack that combines Deep Reinforcement Learning (DRL) and a probabilistic localization method. To test the efficacy of these systems, we prototyped a low-cost mobile robot that we call Covy. Covy can be used for applications such as promoting social distancing during pandemics or estimating the density of a crowd. We evaluated Covy's performance through extensive sets of experiments both in simulated and realistic environments. Our results show that Covy's compound vision algorithm doubles the range of the used depth camera, and its hybrid navigation stack is more robust than a pure DRL-based one.