mlsanityData to 7 Oct 2026

Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challenge

arXiv 2405.16464 ↗First posted 26 May 2024cs.RO cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

This technical report presents the 1st winning model for UG2+, a task in CVPR 2024 UAV Tracking and Pose-Estimation Challenge. This challenge faces difficulties in drone detection, UAV-type classification and 2D/3D trajectory estimation in extreme weather conditions with multi-modal sensor information, including stereo vision, various Lidars, Radars, and audio arrays. Leveraging this information, we propose a multi-modal UAV detection, classification, and 3D tracking method for accurate UAV classification and tracking. A novel classification pipeline which incorporates sequence fusion, region of interest (ROI) cropping, and keyframe selection is proposed. Our system integrates cutting-edge classification techniques and sophisticated post-processing steps to boost accuracy and robustness. The designed pose estimation pipeline incorporates three modules: dynamic points analysis, a multi-object tracker, and trajectory completion techniques. Extensive experiments have validated the effectiveness and precision of our approach. In addition, we also propose a novel dataset pre-processing method and conduct a comprehensive ablation study for our design. We finally achieved the best performance in the classification and tracking of the MMUAD dataset. The code and configuration of our method are available at https://github.com/dtc111111/Multi-Modal-UAV.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 33 of its 61 references that are in the map. First posted on arXiv 26 May 2024.

Builds on

33 of 61 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  4. 2024YOLOv9: Learning What You Want to Learn Using Programmable Gradient Informationcited by 422
  5. 2024SGS-SLAM: Semantic Gaussian Splatting For Neural Dense SLAMcited by 86
  6. 2024Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategycited by 71
  7. 2023MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Trackingcited by 64
  8. 2023SNI-SLAM: Semantic Neural Implicit SLAMcited by 64
  9. 2023VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understandingcited by 62
  10. 2023PLGSLAM: Progressive Neural Scene Represenation with Local to Global Bundle Adjustmentcited by 62
  11. 2023Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networkscited by 54
  12. 2018Contextual Hourglass Network for Semantic Segmentation of High Resolution Aerial Imagerycited by 42
21 more
  1. 2023MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learningcited by 41
  2. 2023RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registrationcited by 37
  3. 2021LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentationcited by 34
  4. 2024Regional Style and Color Transfercited by 30
  5. 2024NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoisingcited by 29
  6. 2019Confidence Trigger Detection: Accelerating Real-time Tracking-by-detection Systemscited by 20
  7. 2024Particle Filter SLAM for Vehicle Localizationcited by 20
  8. 2023MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusioncited by 19
  9. 2024MMAUD: A Comprehensive Multi-Modal Anti-UAV Dataset for Modern Miniature Drone Threatscited by 18
  10. 2020Zero-shot Synthesis with Group-Supervised Learningcited by 17
  11. 2024Enhanced Detection Classification via Clustering SVM for Various Robot Collaboration Taskcited by 17
  12. 2024Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classificationcited by 16
  13. 2024Incremental Joint Learning of Depth, Pose and Implicit Scene Representation on Monocular Camera in Large-scale Scenescited by 15
  14. 2024Adaptive speed planning for Unmanned Vehicle Based on Deep Reinforcement Learningcited by 15
  15. 2023SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detectioncited by 14
  16. 2024Rumor Detection with a novel graph neural network approachcited by 14
  17. 2018Vehicle Pose and Shape Estimation through Multiple Monocular Visioncited by 13
  18. 2024DeepGI: An Automated Approach for Gastrointestinal Tract Segmentation in MRI Scanscited by 13
  19. 2024Localization Through Particle Filter Powered Neural Network Estimated Monocular Camera Posescited by 11
  20. 2021Temporal-MPI: Enabling Multi-Plane Images for Dynamic Scene Modelling via Temporal Basis Learningcited by 8
  21. 2024NGM-SLAM: Gaussian Splatting SLAM with Radiance Field Submapcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 2; 2026: 1.

9 papers.

  1. 2024Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformerscited by 35
  2. 2024Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approachcited by 24
  3. 2024AR Overlay: Training Image Pose Estimation on Curved Surface in a Synthetic Waycited by 15
  4. 2024SAM-Guided Masked Token Prediction for 3D Scene Understandingcited by 3
  5. 2024TAME: Temporal Audio-based Mamba for Enhanced Drone Trajectory Estimation and Classificationcited by 2
  6. 2025UNO: Unified Self-Supervised Monocular Odometry for Platform-Agnostic Deploymentcited by 2
  7. 2024AV-DTEC: Self-Supervised Audio-Visual Fusion for Drone Trajectory Estimation and Classificationcited by 1
  8. 2025Enhancing Low-Altitude Airspace Security: MLLM-Enabled UAV Intent Recognitioncited by 1
  9. 2026Efficient UAV trajectory prediction: A multi-modal deep diffusion frameworkcited by 0