# EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detection

arXiv: https://arxiv.org/abs/2112.11088 · HTML: https://mlsanity.com/paper/2112.11088/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 89 of its 105 references that are in the map.
- First posted on arXiv 21 Dec 2021.

## Builds on (most cited first, 30 of 89)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)

## Built on by (most cited first, 30 of 49)

- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [PointMamba: A Simple State Space Model for Point Cloud Analysis](https://mlsanity.com/paper/2402.10739/) (2024)
- [Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis](https://mlsanity.com/paper/2403.01439/) (2024)
- [Transformation-Equivariant 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2211.11962/) (2022)
- [GraphBEV: Towards Robust BEV Feature Alignment for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2403.11848/) (2024)
- [TR3D: Towards Real-Time Indoor 3D Object Detection](https://mlsanity.com/paper/2302.02858/) (2023)
- [BEVHeight++: Toward Robust Visual Centric 3D Object Detection](https://mlsanity.com/paper/2309.16179/) (2023)
- [Uni3DETR: Unified 3D Detection Transformer](https://mlsanity.com/paper/2310.05699/) (2023)
- [SEED: A Simple and Effective 3D DETR in Point Clouds](https://mlsanity.com/paper/2407.10749/) (2024)
- [DDS3D: Dense Pseudo-Labels with Dynamic Threshold for Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2303.05079/) (2023)
- [VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2401.02702/) (2024)
- [ContrastAlign: Toward Robust BEV Feature Alignment via Contrastive Learning for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2405.16873/) (2024)
- [Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluation](https://mlsanity.com/paper/2310.15676/) (2023)
- [Boosting 3D Object Detection via Object-Focused Image Fusion](https://mlsanity.com/paper/2207.10589/) (2022)
- [Unleash the Potential of Image Branch for Cross-modal 3D Object Detection](https://mlsanity.com/paper/2301.09077/) (2023)
- [Sparse Dense Fusion for 3D Object Detection](https://mlsanity.com/paper/2304.04179/) (2023)
- [StereoDistill: Pick the Cream from LiDAR for Distilling Stereo-based 3D Object Detection](https://mlsanity.com/paper/2301.01615/) (2023)
- [PointSee: Image Enhances Point Cloud](https://mlsanity.com/paper/2211.01664/) (2022)
- [MLF-DET: Multi-Level Fusion for Cross-Modal 3D Object Detection](https://mlsanity.com/paper/2307.09155/) (2023)
- [Diffusion-based 3D Object Detection with Random Boxes](https://mlsanity.com/paper/2309.02049/) (2023)
- [Multi-Modal 3D Object Detection by Box Matching](https://mlsanity.com/paper/2305.07713/) (2023)
- [Anyview: Generalizable Indoor 3D Object Detection with Variable Frames](https://mlsanity.com/paper/2310.05346/) (2023)
- [You Only Look Bottom-Up for Monocular 3D Object Detection](https://mlsanity.com/paper/2401.15319/) (2024)
- [TEM^3-Learning: Time-Efficient Multimodal Multi-Task Learning for Advanced Assistive Driving](https://mlsanity.com/paper/2506.18084/) (2025)
- [UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs](https://mlsanity.com/paper/2511.01768/) (2025)
- [A Generalized Multi-Modal Fusion Detection Framework](https://mlsanity.com/paper/2303.07064/) (2023)
- [3D objects and scenes classification, recognition, segmentation, and reconstruction using 3D point cloud data: A review](https://mlsanity.com/paper/2306.05978/) (2023)
- [Sparse Points to Dense Clouds: Enhancing 3D Detection with Limited LiDAR Data](https://mlsanity.com/paper/2404.06715/) (2024)
- [InScope: A New Real-world 3D Infrastructure-side Collaborative Perception Dataset for Open Traffic Scenarios](https://mlsanity.com/paper/2407.21581/) (2024)
- [All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles](https://mlsanity.com/paper/2510.26641/) (2025)

## Abstract (arXiv)

Recently, fusing the LiDAR point cloud and camera image to improve the performance and robustness of 3D object detection has received more and more attention, as these two modalities naturally possess strong complementarity. In this paper, we propose EPNet++ for multi-modal 3D object detection by introducing a novel Cascade Bi-directional Fusion~(CB-Fusion) module and a Multi-Modal Consistency~(MC) loss. More concretely, the proposed CB-Fusion module enhances point features with plentiful semantic information absorbed from the image features in a cascade bi-directional interaction fusion manner, leading to more powerful and discriminative feature representations. The MC loss explicitly guarantees the consistency between predicted scores from two modalities to obtain more comprehensive and reliable confidence scores. The experimental results on the KITTI, JRDB and SUN-RGBD datasets demonstrate the superiority of EPNet++ over the state-of-the-art methods. Besides, we emphasize a critical but easily overlooked problem, which is to explore the performance and robustness of a 3D detector in a sparser scene. Extensive experiments present that EPNet++ outperforms the existing SOTA methods with remarkable margins in highly sparse point cloud cases, which might be an available direction to reduce the expensive cost of LiDAR sensors. Code is available at: https://github.com/happinesslz/EPNetV2.