A Guide to Image and Video based Small Object Detection using Deep Learning : Case Study of Maritime Surveillance
Not on the map. Why?
Abstract (arXiv)
Small object detection (SOD) in optical images and videos is a challenging problem that even state-of-the-art generic object detection methods fail to accurately localize and identify such objects. Typically, small objects appear in real-world due to large camera-object distance. Because small objects occupy only a small area in the input image (e.g., less than 10%), the information extracted from such a small area is not always rich enough to support decision making. Multidisciplinary strategies are being developed by researchers working at the interface of deep learning and computer vision to enhance the performance of SOD deep learning based methods. In this paper, we provide a comprehensive review of over 160 research papers published between 2017 and 2022 in order to survey this growing subject. This paper summarizes the existing literature and provide a taxonomy that illustrates the broad picture of current research. We investigate how to improve the performance of small object detection in maritime environments, where increasing performance is critical. By establishing a connection between generic and maritime SOD research, future directions have been identified. In addition, the popular datasets that have been used for SOD for generic and maritime applications are discussed, and also well-known evaluation metrics for the state-of-the-art methods on some of the datasets are provided.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 140 of its 313 references that are in the map. First posted on arXiv 26 Jul 2022.
Builds on
140 of 313 references.
References · 140
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2020End-to-End Object Detection with Transformerscited by 7,384
- 2017Mask R-CNNcited by 7,259
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
38 more
- 2016Feature Pyramid Networks for Object Detectioncited by 5,021
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
- 2017Squeeze-and-Excitation Networkscited by 4,763
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
- 2015SSD: Single Shot MultiBox Detectorcited by 3,125
- 2015Fast R-CNNcited by 2,930
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
- 2018YOLOv3: An Incremental Improvementcited by 2,652
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
- 2018CBAM: Convolutional Block Attention Modulecited by 2,463
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
- 2019Searching for MobileNetV3cited by 1,984
- 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
- 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
- 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
- 2017Deformable Convolutional Networkscited by 1,481
- 2016YOLO9000: Better, Faster, Strongercited by 1,429
- 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
- 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
- 2017Random Erasing Data Augmentationcited by 1,235
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
- 2019Objects as Pointscited by 1,172
- 2020Center-based 3D Object Detection and Trackingcited by 1,143
- 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
- 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
- 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
- 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
- 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
- 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
- 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 622
- 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 539
- 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 428
- 2016R-FCN: Object Detection via Region-based Fully Convolutional Networkscited by 414
- 2019Deep Learning for Image Super-resolution: A Surveycited by 368
50 most cited of 140 All 140 (JSON)
Built on by
Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 4; 2026: 5.
14 papers.
Sort:
By area:
- 2023Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Artcited by 15
- 20221st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Resultscited by 14
- 2023MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Resultscited by 7
- 2024Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentionscited by 4
- 2026HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Frameworkcited by 3
- 2025MVQA-68K: A Multi-dimensional and Causally-annotated Dataset with Quality Interpretability for Video Assessmentcited by 2
- 2023Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisonscited by 1
- 2025MVTD: A Benchmark Dataset for Maritime Visual Object Trackingcited by 1
- 2026WUTDet: A 100K-Scale Ship Detection Dataset and Benchmarks with Dense Small Objectscited by 1
- 2026WD-FQDet: Multispectral Detection Transformer via Wavelet Decomposition and Frequency-aware Query Learningcited by 1
- 2025An Empirical Study of Methods for Small Object Detection from Satellite Imagerycited by 0
- 2025LSFDNet: A Single-Stage Fusion and Detection Network for Ships Using SWIR and LWIRcited by 0
- 2026C2FXNet: Coarse-to-Fine Scene Expert for Unified Object Detection across Adverse Weather22 Sep 2026
- 2026GSO-Net: Visual State Machines for Hazardous Freight Transfer Compliance at Petrochemical Logistics Nodes11 Sep 2026
- 2026WD-FQDet: Multispectral Detection Transformer via Wavelet Decomposition and Frequency-aware Query Learning13 May 2026
- 2026WUTDet: A 100K-Scale Ship Detection Dataset and Benchmarks with Dense Small Objects9 Apr 2026
- 2026HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Framework27 Feb 2026
- 2025MVQA-68K: A Multi-dimensional and Causally-annotated Dataset with Quality Interpretability for Video Assessment15 Sep 2025
- 2025LSFDNet: A Single-Stage Fusion and Detection Network for Ships Using SWIR and LWIR28 Jul 2025
- 2025MVTD: A Benchmark Dataset for Maritime Visual Object Tracking3 Jun 2025
- 2025An Empirical Study of Methods for Small Object Detection from Satellite Imagery5 Feb 2025
- 2024Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentions27 Jul 2024
- 2023Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisons14 Nov 2023
- 2023Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Art10 Sep 2023
- 2023Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Artcited by 15
- 20221st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Resultscited by 14
- 2023MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Resultscited by 7
- 2025MVQA-68K: A Multi-dimensional and Causally-annotated Dataset with Quality Interpretability for Video Assessmentcited by 2
- 2023Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisonscited by 1
- 2025MVTD: A Benchmark Dataset for Maritime Visual Object Trackingcited by 1
- 2026WUTDet: A 100K-Scale Ship Detection Dataset and Benchmarks with Dense Small Objectscited by 1
- 2026WD-FQDet: Multispectral Detection Transformer via Wavelet Decomposition and Frequency-aware Query Learningcited by 1
- 2025An Empirical Study of Methods for Small Object Detection from Satellite Imagerycited by 0
- 2025LSFDNet: A Single-Stage Fusion and Detection Network for Ships Using SWIR and LWIRcited by 0
- 2026GSO-Net: Visual State Machines for Hazardous Freight Transfer Compliance at Petrochemical Logistics Nodescited by 0
- 2026GSO-Net: Visual State Machines for Hazardous Freight Transfer Compliance at Petrochemical Logistics Nodes11 Sep 2026
- 2026WD-FQDet: Multispectral Detection Transformer via Wavelet Decomposition and Frequency-aware Query Learning13 May 2026
- 2026WUTDet: A 100K-Scale Ship Detection Dataset and Benchmarks with Dense Small Objects9 Apr 2026
- 2025MVQA-68K: A Multi-dimensional and Causally-annotated Dataset with Quality Interpretability for Video Assessment15 Sep 2025
- 2025LSFDNet: A Single-Stage Fusion and Detection Network for Ships Using SWIR and LWIR28 Jul 2025
- 2025MVTD: A Benchmark Dataset for Maritime Visual Object Tracking3 Jun 2025
- 2025An Empirical Study of Methods for Small Object Detection from Satellite Imagery5 Feb 2025
- 2023Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisons14 Nov 2023
- 2023Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Art10 Sep 2023
- 2023MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results18 Jul 2023
- 20221st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results24 Nov 2022