# Contrastive Learning through Auxiliary Branch for Video Object Detection

arXiv: https://arxiv.org/abs/2508.20551 · HTML: https://mlsanity.com/paper/2508.20551/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 34 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [Simultaneous Deep Transfer Across Domains and Tasks](https://mlsanity.com/paper/1510.02192/) (2015)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Single-Stage Semantic Segmentation from Image Labels](https://mlsanity.com/paper/2005.08104/) (2020)
- [Knowledge Distillation Meets Self-Supervision](https://mlsanity.com/paper/2006.07114/) (2020)
- [TCLR: Temporal Contrastive Learning for Video Representation](https://mlsanity.com/paper/2101.07974/) (2021)
- [Memory Enhanced Global-Local Aggregation for Video Object Detection](https://mlsanity.com/paper/2003.12063/) (2020)
- [Adapting Auxiliary Losses Using Gradient Similarity](https://mlsanity.com/paper/1812.02224/) (2018)
- [Sequence Level Semantics Aggregation for Video Object Detection](https://mlsanity.com/paper/1907.06390/) (2019)
- [Relation Distillation Networks for Video Object Detection](https://mlsanity.com/paper/1908.09511/) (2019)
- [Seq-NMS for Video Object Detection](https://mlsanity.com/paper/1602.08465/) (2016)
- [Object Detection in Video with Spatiotemporal Sampling Networks](https://mlsanity.com/paper/1803.05549/) (2018)
- [Temporal RoI Align for Video Object Recognition](https://mlsanity.com/paper/2109.03495/) (2021)
- [Unsupervised Learning from Video with Deep Neural Embeddings](https://mlsanity.com/paper/1905.11954/) (2019)
- [PTSEFormer: Progressive Temporal-Spatial Enhanced TransFormer Towards Video Object Detection](https://mlsanity.com/paper/2209.02242/) (2022)
- [Objects do not disappear: Video object detection by single-frame object location anticipation](https://mlsanity.com/paper/2308.04770/) (2023)
- [BoxMask: Revisiting Bounding Box Supervision for Video Object Detection](https://mlsanity.com/paper/2210.06008/) (2022)

## Abstract (arXiv)

Video object detection is a challenging task because videos often suffer from image deterioration such as motion blur, occlusion, and deformable shapes, making it significantly more difficult than detecting objects in still images. Prior approaches have improved video object detection performance by employing feature aggregation and complex post-processing techniques, though at the cost of increased computational demands. To improve robustness to image degradation without additional computational load during inference, we introduce a straightforward yet effective Contrastive Learning through Auxiliary Branch (CLAB) method. First, we implement a constrastive auxiliary branch using a contrastive loss to enhance the feature representation capability of the video object detector's backbone. Next, we propose a dynamic loss weighting strategy that emphasizes auxiliary feature learning early in training while gradually prioritizing the detection task as training converges. We validate our approach through comprehensive experiments and ablation studies, demonstrating consistent performance gains. Without bells and whistles, CLAB reaches a performance of 84.0% mAP and 85.2% mAP with ResNet-101 and ResNeXt-101, respectively, on the ImageNet VID dataset, thus achieving state-of-the-art performance for CNN-based models without requiring additional post-processing methods.