# Beyond Skip Connections: Top-Down Modulation for Object Detection

arXiv: https://arxiv.org/abs/1612.06851 · HTML: https://mlsanity.com/paper/1612.06851/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2016.

## Built on by (most cited first, 21 of 21)

- [Embracing Single Stride 3D Object Detector with Sparse Transformer](https://mlsanity.com/paper/2112.06375/) (2021)
- [Towards Large-Scale Small Object Detection: Survey and Benchmarks](https://mlsanity.com/paper/2207.14096/) (2022)
- [QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection](https://mlsanity.com/paper/2103.09136/) (2021)
- [Teaching Matters: Investigating the Role of Supervision in Vision Transformers](https://mlsanity.com/paper/2212.03862/) (2022)
- [GraphFPN: Graph Feature Pyramid Network for Object Detection](https://mlsanity.com/paper/2108.00580/) (2021)
- [Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches](https://mlsanity.com/paper/2206.08016/) (2022)
- [You Should Look at All Objects](https://mlsanity.com/paper/2207.07889/) (2022)
- [CenterNet++ for Object Detection](https://mlsanity.com/paper/2204.08394/) (2022)
- [Boosting Semantic Segmentation from the Perspective of Explicit Class Embeddings](https://mlsanity.com/paper/2308.12894/) (2023)
- [BenchCloudVision: A Benchmark Analysis of Deep Learning Approaches for Cloud Detection and Segmentation in Remote Sensing Imagery](https://mlsanity.com/paper/2402.13918/) (2024)
- [Probabilistic Ranking-Aware Ensembles for Enhanced Object Detections](https://mlsanity.com/paper/2105.03139/) (2021)
- [Trident Pyramid Networks: The importance of processing at the feature pyramid level for better object detection](https://mlsanity.com/paper/2110.04004/) (2021)
- [Colonoscopy polyp detection with massive endoscopic images](https://mlsanity.com/paper/2202.08730/) (2022)
- [Automated Optical Inspection of FAST's Reflector Surface using Drones and Computer Vision](https://mlsanity.com/paper/2212.09039/) (2022)
- [Building artificial neural circuits for domain-general cognition: a primer on brain-inspired systems-level architecture](https://mlsanity.com/paper/2303.13651/) (2023)
- [RMOPP: Robust Multi-Objective Post-Processing for Effective Object Detection](https://mlsanity.com/paper/2102.04582/) (2021)
- [HR-RCNN: Hierarchical Relational Reasoning for Object Detection](https://mlsanity.com/paper/2110.13892/) (2021)
- [Multi-Grid Redundant Bounding Box Annotation for Accurate Object Detection](https://mlsanity.com/paper/2201.01857/) (2022)
- [Multi-Semantic Interactive Learning for Object Detection](https://mlsanity.com/paper/2303.10411/) (2023)
- [Designing High-Performing Networks for Multi-Scale Computer Vision](https://mlsanity.com/paper/2402.12536/) (2024)
- [DPNet: Dynamic Pooling Network for Tiny Object Detection](https://mlsanity.com/paper/2505.02797/) (2025)

## Abstract (arXiv)

In recent years, we have seen tremendous progress in the field of object detection. Most of the recent improvements have been achieved by targeting deeper feedforward networks. However, many hard object categories such as bottle, remote, etc. require representation of fine details and not just coarse, semantic representations. But most of these fine details are lost in the early convolutional layers. What we need is a way to incorporate finer details from lower layers into the detection architecture. Skip connections have been proposed to combine high-level and low-level features, but we argue that selecting the right features from low-level requires top-down contextual information. Inspired by the human visual pathway, in this paper we propose top-down modulations as a way to incorporate fine details into the detection framework. Our approach supplements the standard bottom-up, feedforward ConvNet with a top-down modulation (TDM) network, connected using lateral connections. These connections are responsible for the modulation of lower layer filters, and the top-down network handles the selection and integration of contextual information and low-level features. The proposed TDM architecture provides a significant boost on the COCO testdev benchmark, achieving 28.6 AP for VGG16, 35.2 AP for ResNet101, and 37.3 for InceptionResNetv2 network, without any bells and whistles (e.g., multi-scale, iterative box refinement, etc.).