# Enhanced Radar Perception via Multi-Task Learning: Towards Refined Data for Sensor Fusion Applications

arXiv: https://arxiv.org/abs/2404.06165 · HTML: https://mlsanity.com/paper/2404.06165/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 9 Apr 2024.

## Builds on (most cited first, 15 of 15)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Multi-Task Multi-Sensor Fusion for 3D Object Detection](https://mlsanity.com/paper/2012.12397/) (2020)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection](https://mlsanity.com/paper/2005.07431/) (2020)
- [Depth Estimation from Monocular Images and Sparse radar using Deep Ordinal Regression Network](https://mlsanity.com/paper/2107.07596/) (2021)
- [PixSet : An Opportunity for 3D Computer Vision to Go Beyond Point Clouds With a Full-Waveform LiDAR Dataset](https://mlsanity.com/paper/2102.12010/) (2021)
- [Multi-Task Cross-Modality Attention-Fusion for 2D Object Detection](https://mlsanity.com/paper/2307.08339/) (2023)
- [Motion Classification and Height Estimation of Pedestrians Using Sparse Radar Data](https://mlsanity.com/paper/2103.02278/) (2021)
- [FMCW Radar Height Estimation of Moving Vehicles by Analyzing Multipath Reflections](https://mlsanity.com/paper/2312.11148/) (2023)

## Built on by (most cited first, 5 of 5)

- [CaFNet: A Confidence-Driven Framework for Radar Camera Depth Estimation](https://mlsanity.com/paper/2407.00697/) (2024)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [GET-UP: GEomeTric-aware Depth Estimation with Radar Points UPsampling](https://mlsanity.com/paper/2409.02720/) (2024)
- [LiRCDepth: Lightweight Radar-Camera Depth Estimation via Knowledge Distillation and Uncertainty Guidance](https://mlsanity.com/paper/2412.16380/) (2024)
- [TRIDE: A Text-assisted Radar-Image weather-aware fusion network for Depth Estimation](https://mlsanity.com/paper/2508.08038/) (2025)

## Abstract (arXiv)

Radar and camera fusion yields robustness in perception tasks by leveraging the strength of both sensors. The typical extracted radar point cloud is 2D without height information due to insufficient antennas along the elevation axis, which challenges the network performance. This work introduces a learning-based approach to infer the height of radar points associated with 3D objects. A novel robust regression loss is introduced to address the sparse target challenge. In addition, a multi-task training strategy is employed, emphasizing important features. The average radar absolute height error decreases from 1.69 to 0.25 meters compared to the state-of-the-art height extension method. The estimated target height values are used to preprocess and enrich radar data for downstream perception tasks. Integrating this refined radar information further enhances the performance of existing radar camera fusion models for object detection and depth estimation tasks.