# Context R-CNN: Long Term Temporal Context for Per-Camera Object Detection

arXiv: https://arxiv.org/abs/1912.03538 · HTML: https://mlsanity.com/paper/1912.03538/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2019.

## Built on by (most cited first, 30 of 45)

- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)
- [Global Tracking Transformers](https://mlsanity.com/paper/2203.13250/) (2022)
- [The iWildCam 2021 Competition Dataset](https://mlsanity.com/paper/2105.03494/) (2021)
- [Temporal Query Networks for Fine-grained Video Understanding](https://mlsanity.com/paper/2104.09496/) (2021)
- [Implicit Motion Handling for Video Camouflaged Object Detection](https://mlsanity.com/paper/2203.07363/) (2022)
- [Focal and Efficient IOU Loss for Accurate Bounding Box Regression](https://mlsanity.com/paper/2101.08158/) (2021)
- [With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition](https://mlsanity.com/paper/2111.01024/) (2021)
- [Role of Human-AI Interaction in Selective Prediction](https://mlsanity.com/paper/2112.06751/) (2021)
- [Perspectives on individual animal identification from biology and computer vision](https://mlsanity.com/paper/2103.00560/) (2021)
- [Instance-wise Occlusion and Depth Orders in Natural Scenes](https://mlsanity.com/paper/2111.14562/) (2021)
- [Regular Polytope Networks](https://mlsanity.com/paper/2103.15632/) (2021)
- [Context Understanding in Computer Vision: A Survey](https://mlsanity.com/paper/2302.05011/) (2023)
- [Exploiting Temporal Relations on Radar Perception for Autonomous Driving](https://mlsanity.com/paper/2204.01184/) (2022)
- [Zero-Query Transfer Attacks on Context-Aware Object Detectors](https://mlsanity.com/paper/2203.15230/) (2022)
- [Focus on the Positives: Self-Supervised Learning for Biodiversity Monitoring](https://mlsanity.com/paper/2108.06435/) (2021)
- [SVIP: Sequence VerIfication for Procedures in Videos](https://mlsanity.com/paper/2112.06447/) (2021)
- [MammAlps: A multi-view video behavior monitoring dataset of wild mammals in the Swiss Alps](https://mlsanity.com/paper/2503.18223/) (2025)
- [Machine Learning and Deep Learning -- A review for Ecologists](https://mlsanity.com/paper/2204.05023/) (2022)
- [SAM-PM: Enhancing Video Camouflaged Object Detection using Spatio-Temporal Attention](https://mlsanity.com/paper/2406.05802/) (2024)
- [Dynamic Curriculum Learning for Great Ape Detection in the Wild](https://mlsanity.com/paper/2205.00275/) (2022)
- [Survey on video anomaly detection in dynamic scenes with moving cameras](https://mlsanity.com/paper/2308.07050/) (2023)
- [Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking](https://mlsanity.com/paper/2206.03666/) (2022)
- [ElephantBook: A Semi-Automated Human-in-the-Loop System for Elephant Re-Identification](https://mlsanity.com/paper/2106.15083/) (2021)
- [Multi-view Tracking, Re-ID, and Social Network Analysis of a Flock of Visually Similar Birds in an Outdoor Aviary](https://mlsanity.com/paper/2212.00266/) (2022)
- [Context-Driven Detection of Invertebrate Species in Deep-Sea Video](https://mlsanity.com/paper/2206.00718/) (2022)
- [Fast spline detection in high density microscopy data](https://mlsanity.com/paper/2301.04460/) (2023)
- [ConTra: (Con)text (Tra)nsformer for Cross-Modal Video Retrieval](https://mlsanity.com/paper/2210.04341/) (2022)
- [Erasure-based Interaction Network for RGBT Video Object Detection and A Unified Benchmark](https://mlsanity.com/paper/2308.01630/) (2023)
- [Deep learning-based ecological analysis of camera trap images is impacted by training data quality and quantity](https://mlsanity.com/paper/2408.14348/) (2024)
- [VoCap: Video Object Captioning and Segmentation from Any Prompt](https://mlsanity.com/paper/2508.21809/) (2025)

## Abstract (arXiv)

In static monitoring cameras, useful contextual information can stretch far beyond the few seconds typical video understanding models might see: subjects may exhibit similar behavior over multiple days, and background objects remain static. Due to power and storage constraints, sampling frequencies are low, often no faster than one frame per second, and sometimes are irregular due to the use of a motion trigger. In order to perform well in this setting, models must be robust to irregular sampling rates. In this paper we propose a method that leverages temporal context from the unlabeled frames of a novel camera to improve performance at that camera. Specifically, we propose an attention-based approach that allows our model, Context R-CNN, to index into a long term memory bank constructed on a per-camera basis and aggregate contextual features from other frames to boost object detection performance on the current frame. We apply Context R-CNN to two settings: (1) species detection using camera traps, and (2) vehicle detection in traffic cameras, showing in both settings that Context R-CNN leads to performance gains over strong baselines. Moreover, we show that increasing the contextual time horizon leads to improved results. When applied to camera trap data from the Snapshot Serengeti dataset, Context R-CNN with context from up to a month of images outperforms a single-frame baseline by 17.9% mAP, and outperforms S3D (a 3d convolution based baseline) by 11.2% mAP.