# ActiveAnno3D -- An Active Learning Framework for Multi-Modal 3D Object Detection

arXiv: https://arxiv.org/abs/2402.03235 · HTML: https://mlsanity.com/paper/2402.03235/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 30 of 32)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2012.13755/) (2020)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Deep Active Learning for Efficient Training of a LiDAR 3D Object Detector](https://mlsanity.com/paper/1901.10609/) (2019)
- [3D BAT: A Semi-Automatic, Web-based 3D Annotation Toolbox for Full-Surround, Multi-Modal Data Streams](https://mlsanity.com/paper/1905.00525/) (2019)
- [Trajectory Prediction in Autonomous Driving with a Lane Heading Auxiliary Loss](https://mlsanity.com/paper/2011.06679/) (2020)
- [Exploring Active 3D Object Detection from a Generalization Perspective](https://mlsanity.com/paper/2301.09249/) (2023)
- [InfraDet3D: Multi-Modal 3D Object Detection based on Roadside Infrastructure Camera and LiDAR Sensors](https://mlsanity.com/paper/2305.00314/) (2023)
- [Entropic Open-set Active Learning](https://mlsanity.com/paper/2312.14126/) (2023)
- [Bi3D: Bi-domain Active Learning for Cross-domain 3D Object Detection](https://mlsanity.com/paper/2303.05886/) (2023)
- [Providentia -- A Large-Scale Sensor System for the Assistance of Autonomous Vehicles and Its Evaluation](https://mlsanity.com/paper/1906.06789/) (2019)
- [The Why, When, and How to Use Active Learning in Large-Data-Driven 3D Object Detection for Safe Autonomous Driving: An Empirical Exploration](https://mlsanity.com/paper/2401.16634/) (2024)
- [Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2205.07708/) (2022)
- [Salient Sign Detection In Safe Autonomous Driving: AI Which Reasons Over Full Visual Context](https://mlsanity.com/paper/2301.05804/) (2023)
- [On Salience-Sensitive Sign Classification in Autonomous Vehicle Path Planning: Experimental Explorations with a Novel Dataset](https://mlsanity.com/paper/2112.00942/) (2021)
- [KECOR: Kernel Coding Rate Maximization for Active 3D Object Detection](https://mlsanity.com/paper/2307.07942/) (2023)
- [TUMTraf Event: Calibration and Fusion Resulting in a Dataset for Roadside Event-Based and RGB Cameras](https://mlsanity.com/paper/2401.08474/) (2024)
- [Robust Traffic Light Detection Using Salience-Sensitive Loss: Computational Framework and Evaluations](https://mlsanity.com/paper/2305.04516/) (2023)
- [A Survey of Robust 3D Object Detection Methods in Point Clouds](https://mlsanity.com/paper/2204.00106/) (2022)
- [Monocular 3D Object Detection with LiDAR Guided Semi Supervised Active Learning](https://mlsanity.com/paper/2307.08415/) (2023)
- [Real-Time And Robust 3D Object Detection with Roadside LiDARs](https://mlsanity.com/paper/2207.05200/) (2022)
- [Navya3DSeg -- Navya 3D Semantic Segmentation Dataset & split generation for autonomous vehicles](https://mlsanity.com/paper/2302.08292/) (2023)
- [Multi-Task Consistency for Active Learning](https://mlsanity.com/paper/2306.12398/) (2023)
- [BaSAL: Size-Balanced Warm Start Active Learning for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2310.08035/) (2023)

## Built on by (most cited first, 17 of 17)

- [Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Sets](https://mlsanity.com/paper/2402.07320/) (2024)
- [V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detection](https://mlsanity.com/paper/2411.08402/) (2024)
- [Perspective-Invariant 3D Object Detection](https://mlsanity.com/paper/2507.17665/) (2025)
- [Language-Driven Active Learning for Diverse Open-Set 3D Object Detection](https://mlsanity.com/paper/2404.12856/) (2024)
- [Uncertainty Meets Diversity: A Comprehensive Active Learning Framework for Indoor 3D Object Detection](https://mlsanity.com/paper/2503.16125/) (2025)
- [Evaluating Vision-Language Models for Zero-Shot Detection, Classification, and Association of Motorcycles, Passengers, and Helmets](https://mlsanity.com/paper/2408.02244/) (2024)
- [PointCompress3D: A Point Cloud Compression Framework for Roadside LiDARs in Intelligent Transportation Systems](https://mlsanity.com/paper/2405.01750/) (2024)
- [Perception Without Vision for Trajectory Prediction: Ego Vehicle Dynamics as Scene Representation for Efficient Active Learning in Autonomous Driving](https://mlsanity.com/paper/2405.09049/) (2024)
- [Evaluating Cascaded Methods of Vision-Language Models for Zero-Shot Detection and Association of Hardhats for Increased Construction Safety](https://mlsanity.com/paper/2410.12225/) (2024)
- [Towards Vision Zero: The TUM Traffic Accid3nD Dataset](https://mlsanity.com/paper/2503.12095/) (2025)
- [Automated Data Curation Using GPS & NLP to Generate Instruction-Action Pairs for Autonomous Vehicle Vision-Language Navigation Datasets](https://mlsanity.com/paper/2505.03174/) (2025)
- [Distribution Discrepancy and Feature Heterogeneity for Active 3D Object Detection](https://mlsanity.com/paper/2409.05425/) (2024)
- [Enhancing Highway Safety: Accident Detection on the A9 Test Stretch Using Roadside Sensors](https://mlsanity.com/paper/2502.00402/) (2025)
- [Inconsistency-based Active Learning for LiDAR Object Detection](https://mlsanity.com/paper/2505.00511/) (2025)
- [Safety-Critical Learning for Long-Tail Events: The TUM Traffic Accident Dataset](https://mlsanity.com/paper/2508.14567/) (2025)
- [Finding the Reflection Point: Unpadding Images to Remove Data Augmentation Artifacts in Large Open Source Image Datasets for Machine Learning](https://mlsanity.com/paper/2504.03168/) (2025)
- [HeAL3D: Heuristical-enhanced Active Learning for 3D Object Detection](https://mlsanity.com/paper/2505.00507/) (2025)

## Abstract (arXiv)

The curation of large-scale datasets is still costly and requires much time and resources. Data is often manually labeled, and the challenge of creating high-quality datasets remains. In this work, we fill the research gap using active learning for multi-modal 3D object detection. We propose ActiveAnno3D, an active learning framework to select data samples for labeling that are of maximum informativeness for training. We explore various continuous training methods and integrate the most efficient method regarding computational demand and detection performance. Furthermore, we perform extensive experiments and ablation studies with BEVFusion and PV-RCNN on the nuScenes and TUM Traffic Intersection dataset. We show that we can achieve almost the same performance with PV-RCNN and the entropy-based query strategy when using only half of the training data (77.25 mAP compared to 83.50 mAP) of the TUM Traffic Intersection dataset. BEVFusion achieved an mAP of 64.31 when using half of the training data and 75.0 mAP when using the complete nuScenes dataset. We integrate our active learning framework into the proAnno labeling tool to enable AI-assisted data selection and labeling and minimize the labeling costs. Finally, we provide code, weights, and visualization results on our website: https://active3d-framework.github.io/active3d-framework.