# Robust 3D Object Detection from LiDAR-Radar Point Clouds via Cross-Modal Feature Augmentation

arXiv: https://arxiv.org/abs/2309.17336 · HTML: https://mlsanity.com/paper/2309.17336/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [Cross Modal Distillation for Supervision Transfer](https://mlsanity.com/paper/1507.00448/) (2015)
- [Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds](https://mlsanity.com/paper/2203.11139/) (2022)
- [TANet: Robust 3D Object Detection from Point Clouds with Triple Attention](https://mlsanity.com/paper/1912.05163/) (2019)
- [X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning](https://mlsanity.com/paper/2203.00843/) (2022)
- [RADDet: Range-Azimuth-Doppler based Radar Object Detection for Dynamic Road Users](https://mlsanity.com/paper/2105.00363/) (2021)
- [HVPR: Hybrid Voxel-Point Representation for Single-stage 3D Object Detection](https://mlsanity.com/paper/2104.00902/) (2021)
- [Learning Cross-Modal Deep Representations for Robust Pedestrian Detection](https://mlsanity.com/paper/1704.02431/) (2017)
- [Pointillism: Accurate 3D bounding box estimation with multi-radars](https://mlsanity.com/paper/2203.04440/) (2022)
- [Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embedding](https://mlsanity.com/paper/1611.06638/) (2016)
- [Boosting 3D Object Detection by Simulating Multimodality on Point Clouds](https://mlsanity.com/paper/2206.14971/) (2022)
- [DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination](https://mlsanity.com/paper/1909.07231/) (2019)

## Built on by (most cited first, 12 of 12)

- [HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning](https://mlsanity.com/paper/2407.15680/) (2024)
- [RadarPillars: Efficient Object Detection from 4D Radar Point Clouds](https://mlsanity.com/paper/2408.05020/) (2024)
- [Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2312.04861/) (2023)
- [ZFusion: An Effective Fuser of Camera and 4D Radar for 3D Object Perception in Autonomous Driving](https://mlsanity.com/paper/2504.03438/) (2025)
- [MoRAL: Motion-aware Multi-Frame 4D Radar and LiDAR Fusion for Robust 3D Object Detection](https://mlsanity.com/paper/2505.09422/) (2025)
- [TARS: Traffic-Aware Radar Scene Flow Estimation](https://mlsanity.com/paper/2503.10210/) (2025)
- [All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles](https://mlsanity.com/paper/2510.26641/) (2025)
- [SCKD: Semi-Supervised Cross-Modality Knowledge Distillation for 4D Radar Object Detection](https://mlsanity.com/paper/2412.14571/) (2024)
- [ELMAR: Enhancing LiDAR Detection with 4D Radar Motion Awareness and Cross-modal Uncertainty](https://mlsanity.com/paper/2506.17958/) (2025)
- [RadarXFormer: Robust Object Detection via Cross-Dimension Fusion of 4D Radar Spectra and Images for Autonomous Driving](https://mlsanity.com/paper/2603.14822/) (2026)
- [LEO: Graph Attention Network based Hybrid Multi Sensor Extended Object Fusion and Tracking for Autonomous Driving Applications](https://mlsanity.com/paper/2604.02206/) (2026)
- [ESAFusion: LiDAR--4-D Radar Fusion via Local Geometric Complementation and Multiscale Adaptive Interaction for 3-D Object Detection](https://mlsanity.com/paper/2609.14619/) (2026)

## Abstract (arXiv)

This paper presents a novel framework for robust 3D object detection from point clouds via cross-modal hallucination. Our proposed approach is agnostic to either hallucination direction between LiDAR and 4D radar. We introduce multiple alignments on both spatial and feature levels to achieve simultaneous backbone refinement and hallucination generation. Specifically, spatial alignment is proposed to deal with the geometry discrepancy for better instance matching between LiDAR and radar. The feature alignment step further bridges the intrinsic attribute gap between the sensing modalities and stabilizes the training. The trained object detection models can deal with difficult detection cases better, even though only single-modal data is used as the input during the inference stage. Extensive experiments on the View-of-Delft (VoD) dataset show that our proposed method outperforms the state-of-the-art (SOTA) methods for both radar and LiDAR object detection while maintaining competitive efficiency in runtime. Code is available at https://github.com/DJNing/See_beyond_seeing.