# C^3Net: End-to-End deep learning for efficient real-time visual active camera control

arXiv: https://arxiv.org/abs/2107.13233 · HTML: https://mlsanity.com/paper/2107.13233/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 28 Jul 2021.

## Builds on (most cited first, 14 of 14)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [Learning non-maximum suppression](https://mlsanity.com/paper/1705.02950/) (2017)
- [Siamese Cascaded Region Proposal Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1812.06148/) (2018)
- [End-to-end Active Object Tracking via Reinforcement Learning](https://mlsanity.com/paper/1705.10561/) (2017)
- [REVAMP2T: Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking](https://mlsanity.com/paper/1911.09217/) (2019)
- [GPU-based Pedestrian Detection for Autonomous Driving](https://mlsanity.com/paper/1611.01642/) (2016)
- [An In-Depth Analysis of Visual Tracking with Siamese Neural Networks](https://mlsanity.com/paper/1707.00569/) (2017)
- [On the Interaction Between Deep Detectors and Siamese Trackers in Video Surveillance](https://mlsanity.com/paper/1910.14552/) (2019)

## Abstract (arXiv)

The need for automated real-time visual systems in applications such as smart camera surveillance, smart environments, and drones necessitates the improvement of methods for visual active monitoring and control. Traditionally, the active monitoring task has been handled through a pipeline of modules such as detection, filtering, and control. However, such methods are difficult to jointly optimize and tune their various parameters for real-time processing in resource constraint systems. In this paper a deep Convolutional Camera Controller Neural Network is proposed to go directly from visual information to camera movement to provide an efficient solution to the active vision problem. It is trained end-to-end without bounding box annotations to control a camera and follow multiple targets from raw pixel values. Evaluation through both a simulation framework and real experimental setup, indicate that the proposed solution is robust to varying conditions and able to achieve better monitoring performance than traditional approaches both in terms of number of targets monitored as well as in effective monitoring time. The advantage of the proposed approach is that it is computationally less demanding and can run at over 10 FPS (~4x speedup) on an embedded smart camera providing a practical and affordable solution to real-time active monitoring.