# FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction

arXiv: https://arxiv.org/abs/1901.03495 · HTML: https://mlsanity.com/paper/1901.03495/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2019.

## Built on by (most cited first, 16 of 16)

- [PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection](https://mlsanity.com/paper/2102.00463/) (2021)
- [CBNet: A Composite Backbone Network Architecture for Object Detection](https://mlsanity.com/paper/2107.00420/) (2021)
- [Vision Transformer with Progressive Sampling](https://mlsanity.com/paper/2108.01684/) (2021)
- [DAMO-YOLO : A Report on Real-Time Object Detection Design](https://mlsanity.com/paper/2211.15444/) (2022)
- [Visual Parser: Representing Part-whole Hierarchies with Transformers](https://mlsanity.com/paper/2107.05790/) (2021)
- [GiraffeDet: A Heavy-Neck Paradigm for Object Detection](https://mlsanity.com/paper/2202.04256/) (2022)
- [MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection](https://mlsanity.com/paper/2111.13336/) (2021)
- [Pedestrian Detection: Domain Generalization, CNNs, Transformers and Beyond](https://mlsanity.com/paper/2201.03176/) (2022)
- [Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Data](https://mlsanity.com/paper/2407.10200/) (2024)
- [Review of Face Presentation Attack Detection Competitions](https://mlsanity.com/paper/2112.11290/) (2021)
- [Nighttime Pedestrian Detection Based on Fore-Background Contrast Learning](https://mlsanity.com/paper/2408.03030/) (2024)
- [Cycled Compositional Learning between Images and Text](https://mlsanity.com/paper/2107.11509/) (2021)
- [Gramian Attention Heads are Strong yet Efficient Vision Learners](https://mlsanity.com/paper/2310.16483/) (2023)
- [CEC-CNN: A Consecutive Expansion-Contraction Convolutional Network for Very Small Resolution Medical Image Classification](https://mlsanity.com/paper/2209.13661/) (2022)
- [Evaluating Supervision Levels Trade-Offs for Infrared-Based People Counting](https://mlsanity.com/paper/2311.11974/) (2023)
- [i-WiViG: Interpretable Window Vision GNN](https://mlsanity.com/paper/2503.08321/) (2025)

## Abstract (arXiv)

The basic principles in designing convolutional neural network (CNN) structures for predicting objects on different levels, e.g., image-level, region-level, and pixel-level are diverging. Generally, network structures designed specifically for image classification are directly used as default backbone structure for other tasks including detection and segmentation, but there is seldom backbone structure designed under the consideration of unifying the advantages of networks designed for pixel-level or region-level predicting tasks, which may require very deep features with high resolution. Towards this goal, we design a fish-like network, called FishNet. In FishNet, the information of all resolutions is preserved and refined for the final task. Besides, we observe that existing works still cannot directly propagate the gradient information from deep layers to shallow layers. Our design can better handle this problem. Extensive experiments have been conducted to demonstrate the remarkable performance of the FishNet. In particular, on ImageNet-1k, the accuracy of FishNet is able to surpass the performance of DenseNet and ResNet with fewer parameters. FishNet was applied as one of the modules in the winning entry of the COCO Detection 2018 challenge. The code is available at https://github.com/kevin-ssy/FishNet.