# The Why, When, and How to Use Active Learning in Large-Data-Driven 3D Object Detection for Safe Autonomous Driving: An Empirical Exploration

arXiv: https://arxiv.org/abs/2401.16634 · HTML: https://mlsanity.com/paper/2401.16634/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 64 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Builds on (most cited first, 30 of 31)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2304.14340/) (2023)
- [Learning how to Active Learn: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1708.02383/) (2017)
- ["Curse of rarity" for autonomous vehicles](https://mlsanity.com/paper/2207.02749/) (2022)
- [Scalable Active Learning for Object Detection](https://mlsanity.com/paper/2004.04699/) (2020)
- [FocalFormer3D : Focusing on Hard Instance for 3D Object Detection](https://mlsanity.com/paper/2308.04556/) (2023)
- [Deep Active Learning for Efficient Training of a LiDAR 3D Object Detector](https://mlsanity.com/paper/1901.10609/) (2019)
- [LaneAF: Robust Multi-Lane Detection with Affinity Fields](https://mlsanity.com/paper/2103.12040/) (2021)
- [3D BAT: A Semi-Automatic, Web-based 3D Annotation Toolbox for Full-Surround, Multi-Modal Data Streams](https://mlsanity.com/paper/1905.00525/) (2019)
- [BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidance and Temporal Aggregation](https://mlsanity.com/paper/2303.17099/) (2023)
- [Trajectory Prediction in Autonomous Driving with a Lane Heading Auxiliary Loss](https://mlsanity.com/paper/2011.06679/) (2020)
- [Bi3D: Bi-domain Active Learning for Cross-domain 3D Object Detection](https://mlsanity.com/paper/2303.05886/) (2023)
- [Safe Active Dynamics Learning and Control: A Sequential Exploration-Exploitation Framework](https://mlsanity.com/paper/2008.11700/) (2020)
- [Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2205.07708/) (2022)
- [Salient Sign Detection In Safe Autonomous Driving: AI Which Reasons Over Full Visual Context](https://mlsanity.com/paper/2301.05804/) (2023)
- [On Salience-Sensitive Sign Classification in Autonomous Vehicle Path Planning: Experimental Explorations with a Novel Dataset](https://mlsanity.com/paper/2112.00942/) (2021)
- [KECOR: Kernel Coding Rate Maximization for Active 3D Object Detection](https://mlsanity.com/paper/2307.07942/) (2023)
- [Leveraging Pre-Trained 3D Object Detection Models For Fast Ground Truth Generation](https://mlsanity.com/paper/1807.06072/) (2018)
- [Robust Traffic Light Detection Using Salience-Sensitive Loss: Computational Framework and Evaluations](https://mlsanity.com/paper/2305.04516/) (2023)
- [Monocular 3D Object Detection with LiDAR Guided Semi Supervised Active Learning](https://mlsanity.com/paper/2307.08415/) (2023)
- [MAP-Gen: An Automated 3D-Box Annotation Flow with Multimodal Attention Point Generator](https://mlsanity.com/paper/2203.15700/) (2022)
- [Navya3DSeg -- Navya 3D Semantic Segmentation Dataset & split generation for autonomous vehicles](https://mlsanity.com/paper/2302.08292/) (2023)
- [Multi-Task Consistency for Active Learning](https://mlsanity.com/paper/2306.12398/) (2023)
- [Co-training for On-board Deep Object Detection](https://mlsanity.com/paper/2008.05534/) (2020)
- [Does Redundancy in AI Perception Systems Help to Test for Super-Human Automated Driving Performance?](https://mlsanity.com/paper/2112.04758/) (2021)
- [CHAMP: Crowdsourced, History-Based Advisory of Mapped Pedestrians for Safer Driver Assistance Systems](https://mlsanity.com/paper/2301.05842/) (2023)

## Built on by (most cited first, 17 of 17)

- [Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving](https://mlsanity.com/paper/2403.19838/) (2024)
- [Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Sets](https://mlsanity.com/paper/2402.07320/) (2024)
- [ActiveAnno3D -- An Active Learning Framework for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2402.03235/) (2024)
- [Language-Driven Active Learning for Diverse Open-Set 3D Object Detection](https://mlsanity.com/paper/2404.12856/) (2024)
- [Evaluating Vision-Language Models for Zero-Shot Detection, Classification, and Association of Motorcycles, Passengers, and Helmets](https://mlsanity.com/paper/2408.02244/) (2024)
- [Driver Activity Classification Using Generalizable Representations from Vision-Language Models](https://mlsanity.com/paper/2404.14906/) (2024)
- [Generalizable Prompt Learning of CLIP: A Brief Overview](https://mlsanity.com/paper/2503.01263/) (2025)
- [Perception Without Vision for Trajectory Prediction: Ego Vehicle Dynamics as Scene Representation for Efficient Active Learning in Autonomous Driving](https://mlsanity.com/paper/2405.09049/) (2024)
- [Advancing Prompt Learning through an External Layer](https://mlsanity.com/paper/2407.19674/) (2024)
- [Towards Vision Zero: The TUM Traffic Accid3nD Dataset](https://mlsanity.com/paper/2503.12095/) (2025)
- [Automated Data Curation Using GPS & NLP to Generate Instruction-Action Pairs for Autonomous Vehicle Vision-Language Navigation Datasets](https://mlsanity.com/paper/2505.03174/) (2025)
- [Enhancing Target-unspecific Tasks through a Features Matrix](https://mlsanity.com/paper/2505.03414/) (2025)
- [To Label or Not to Label: PALM -- A Predictive Model for Evaluating Sample Efficiency in Active Learning Models](https://mlsanity.com/paper/2507.15381/) (2025)
- [Enhancing Highway Safety: Accident Detection on the A9 Test Stretch Using Roadside Sensors](https://mlsanity.com/paper/2502.00402/) (2025)
- [Safety-Critical Learning for Long-Tail Events: The TUM Traffic Accident Dataset](https://mlsanity.com/paper/2508.14567/) (2025)
- [CAPS: Context-Aware Priority Sampling for Enhanced Imitation Learning in Autonomous Driving](https://mlsanity.com/paper/2503.01650/) (2025)
- [HeAL3D: Heuristical-enhanced Active Learning for 3D Object Detection](https://mlsanity.com/paper/2505.00507/) (2025)

## Abstract (arXiv)

Active learning strategies for 3D object detection in autonomous driving datasets may help to address challenges of data imbalance, redundancy, and high-dimensional data. We demonstrate the effectiveness of entropy querying to select informative samples, aiming to reduce annotation costs and improve model performance. We experiment using the BEVFusion model for 3D object detection on the nuScenes dataset, comparing active learning to random sampling and demonstrating that entropy querying outperforms in most cases. The method is particularly effective in reducing the performance gap between majority and minority classes. Class-specific analysis reveals efficient allocation of annotated resources for limited data budgets, emphasizing the importance of selecting diverse and informative data for model training. Our findings suggest that entropy querying is a promising strategy for selecting data that enhances model learning in resource-constrained environments.