# Multi-level and multi-modal feature fusion for accurate 3D object detection in Connected and Automated Vehicles

arXiv: https://arxiv.org/abs/2212.07560 · HTML: https://mlsanity.com/paper/2212.07560/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 15 Dec 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud](https://mlsanity.com/paper/1710.07368/) (2017)
- [MVX-Net: Multimodal VoxelNet for 3D Object Detection](https://mlsanity.com/paper/1904.01649/) (2019)
- [LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving](https://mlsanity.com/paper/1903.08701/) (2019)
- [Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review](https://mlsanity.com/paper/2004.05224/) (2020)
- [Vehicle Detection from 3D Lidar Using Fully Convolutional Network](https://mlsanity.com/paper/1608.07916/) (2016)
- [SNIPER: Efficient Multi-Scale Training](https://mlsanity.com/paper/1805.09300/) (2018)
- [Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection](https://mlsanity.com/paper/2109.02499/) (2021)
- [Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation](https://mlsanity.com/paper/1904.11466/) (2019)
- [Bag of Freebies for Training Object Detection Neural Networks](https://mlsanity.com/paper/1902.04103/) (2019)
- [Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds](https://mlsanity.com/paper/1904.07537/) (2019)
- [3D Object Detection Using Scale Invariant and Feature Reweighting Networks](https://mlsanity.com/paper/1901.02237/) (2019)
- [3D Bounding Box Estimation for Autonomous Vehicles by Cascaded Geometric Constraints and Depurated 2D Detections Using 3D Results](https://mlsanity.com/paper/1909.01867/) (2019)
- [Traffic-Net: 3D Traffic Monitoring Using a Single Camera](https://mlsanity.com/paper/2109.09165/) (2021)
- [Feature Intertwiner for Object Detection](https://mlsanity.com/paper/1903.11851/) (2019)

## Built on by (most cited first, 1 of 1)

- [Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Models](https://mlsanity.com/paper/2606.13840/) (2026)

## Abstract (arXiv)

Aiming at highly accurate object detection for connected and automated vehicles (CAVs), this paper presents a Deep Neural Network based 3D object detection model that leverages a three-stage feature extractor by developing a novel LIDAR-Camera fusion scheme. The proposed feature extractor extracts high-level features from two input sensory modalities and recovers the important features discarded during the convolutional process. The novel fusion scheme effectively fuses features across sensory modalities and convolutional layers to find the best representative global features. The fused features are shared by a two-stage network: the region proposal network (RPN) and the detection head (DH). The RPN generates high-recall proposals, and the DH produces final detection results. The experimental results show the proposed model outperforms more recent research on the KITTI 2D and 3D detection benchmark, particularly for distant and highly occluded instances.