# Multi-View Radar Semantic Segmentation

arXiv: https://arxiv.org/abs/2103.16214 · HTML: https://mlsanity.com/paper/2103.16214/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 57 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset](https://mlsanity.com/paper/1909.01300/) (2019)
- [CARRADA Dataset: Camera and Automotive Radar with Range-Angle-Doppler Annotations](https://mlsanity.com/paper/2005.01456/) (2020)
- [RODNet: A Real-Time Radar Object Detection Network Cross-Supervised by Camera-Radar Fused Object 3D Localization](https://mlsanity.com/paper/2102.05150/) (2021)
- [CNN based Road User Detection using the 3D Radar Cube](https://mlsanity.com/paper/2004.12165/) (2020)
- [Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar](https://mlsanity.com/paper/1912.06613/) (2019)
- [2D Car Detection in Radar Data with PointNets](https://mlsanity.com/paper/1904.08414/) (2019)
- [RAMP-CNN: A Novel Neural Network for Enhanced Automotive Radar Object Recognition](https://mlsanity.com/paper/2011.08981/) (2020)
- [Probabilistic Oriented Object Detection in Automotive Radar](https://mlsanity.com/paper/2004.05310/) (2020)
- [Deep Radar Detector](https://mlsanity.com/paper/1906.12187/) (2019)
- [RSS-Net: Weakly-Supervised Multi-Class Semantic Segmentation with FMCW Radar](https://mlsanity.com/paper/2004.03451/) (2020)
- [Object Detection and 3D Estimation via an FMCW Radar Using a Fully Convolutional Network](https://mlsanity.com/paper/1902.05394/) (2019)
- [Experiments with mmWave Automotive Radar Test-bed](https://mlsanity.com/paper/1912.12566/) (2019)
- [Deep Open Space Segmentation using Automotive Radar](https://mlsanity.com/paper/2004.03449/) (2020)
- [TinyRadarNN: Combining Spatial and Temporal Convolutional Neural Networks for Embedded Gesture Recognition with Short Range Radars](https://mlsanity.com/paper/2006.16281/) (2020)
- [Vehicle classification based on convolutional networks applied to FM-CW radar signals](https://mlsanity.com/paper/1710.05718/) (2017)
- [Low-cost low-power in-vehicle occupant detection with mm-wave FMCW radar](https://mlsanity.com/paper/1908.04417/) (2019)
- [MmWave Radar Point Cloud Segmentation using GMM in Multimodal Traffic Monitoring](https://mlsanity.com/paper/1911.06364/) (2019)

## Built on by (most cited first, 30 of 34)

- [SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2204.03636/) (2022)
- [Raw High-Definition Radar for Multi-Task Learning](https://mlsanity.com/paper/2112.10646/) (2021)
- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [T-FFTRadNet: Object Detection with Swin Vision Transformers from Raw ADC Radar Signals](https://mlsanity.com/paper/2303.16940/) (2023)
- [Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2312.04861/) (2023)
- [Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar](https://mlsanity.com/paper/2405.04662/) (2024)
- [OpenForest: A data catalogue for machine learning in forest monitoring](https://mlsanity.com/paper/2311.00277/) (2023)
- [A recurrent CNN for online object detection on raw radar frames](https://mlsanity.com/paper/2212.11172/) (2022)
- [TransRadar: Adaptive-Directional Transformer for Real-Time Multi-View Radar Semantic Segmentation](https://mlsanity.com/paper/2310.02260/) (2023)
- [Towards 3D Scene Understanding by Referring Synthetic Models](https://mlsanity.com/paper/2203.10546/) (2022)
- [RETR: Multi-View Radar Detection Transformer for Indoor Perception](https://mlsanity.com/paper/2411.10293/) (2024)
- [Towards Foundational Models for Single-Chip Radar](https://mlsanity.com/paper/2509.12482/) (2025)
- [Self-Supervised Velocity Estimation for Automotive Radar Object Detection Networks](https://mlsanity.com/paper/2207.03146/) (2022)
- [SparseRadNet: Sparse Perception Neural Network on Subsampled Radar Data](https://mlsanity.com/paper/2406.10600/) (2024)
- [4D Radar Semantic Segmentation of People in Field Conditions Using Temporal Multi-View Networks](https://mlsanity.com/paper/2404.05307/) (2024)
- [SIRA: Scalable Inter-frame Relation and Association for Radar Perception](https://mlsanity.com/paper/2411.02220/) (2024)
- [EchoONE: Segmenting Multiple echocardiography Planes in One Model](https://mlsanity.com/paper/2412.02993/) (2024)
- [Semantic Information Marketing in The Metaverse: A Learning-Based Contract Theory Framework](https://mlsanity.com/paper/2302.11457/) (2023)
- [Towards more precise automatic analysis: a comprehensive survey of deep learning-based multi-organ segmentation](https://mlsanity.com/paper/2303.00232/) (2023)
- [RadarFormer: Lightweight and Accurate Real-Time Radar Object Detection Model](https://mlsanity.com/paper/2304.08447/) (2023)
- [Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training](https://mlsanity.com/paper/2309.16956/) (2023)
- [ERASE-Net: Efficient Segmentation Networks for Automotive Radar Signals](https://mlsanity.com/paper/2209.12940/) (2022)
- [Improving Extrinsics between RADAR and LIDAR using Learning](https://mlsanity.com/paper/2305.10594/) (2023)
- [OCHID-Fi: Occlusion-Robust Hand Pose Estimation in 3D via RF-Vision](https://mlsanity.com/paper/2308.10146/) (2023)
- [CaRaFFusion: Improving 2D Semantic Segmentation with Camera-Radar Point Cloud Fusion and Zero-Shot Image Inpainting](https://mlsanity.com/paper/2505.03679/) (2025)
- [Controllable Radar Simulation with Waveform Parameter Embedding](https://mlsanity.com/paper/2506.03134/) (2025)
- [RADE-Net: Robust Attention Network for Radar-Only Object Detection in Adverse Weather](https://mlsanity.com/paper/2602.19994/) (2026)
- [Deep learning for radar data exploitation of autonomous vehicle](https://mlsanity.com/paper/2203.08038/) (2022)
- [A survey on deep learning approaches for data integration in autonomous driving system](https://mlsanity.com/paper/2306.11740/) (2023)
- [Leveraging Self-Supervised Instance Contrastive Learning for Radar Object Detection](https://mlsanity.com/paper/2402.08427/) (2024)

## Abstract (arXiv)

Understanding the scene around the ego-vehicle is key to assisted and autonomous driving. Nowadays, this is mostly conducted using cameras and laser scanners, despite their reduced performances in adverse weather conditions. Automotive radars are low-cost active sensors that measure properties of surrounding objects, including their relative speed, and have the key advantage of not being impacted by rain, snow or fog. However, they are seldom used for scene understanding due to the size and complexity of radar raw data and the lack of annotated datasets. Fortunately, recent open-sourced datasets have opened up research on classification, object detection and semantic segmentation with raw radar signals using end-to-end trainable models. In this work, we propose several novel architectures, and their associated losses, which analyse multiple "views" of the range-angle-Doppler radar tensor to segment it semantically. Experiments conducted on the recent CARRADA dataset demonstrate that our best model outperforms alternative models, derived either from the semantic segmentation of natural images or from radar scene understanding, while requiring significantly fewer parameters. Both our code and trained models are available at https://github.com/valeoai/MVRSS.