# Cross-modal semantic segmentation for indoor environmental perception using single-chip millimeter-wave radar raw data

arXiv: https://arxiv.org/abs/2411.00499 · HTML: https://mlsanity.com/paper/2411.00499/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter](https://mlsanity.com/paper/2010.08196/) (2020)
- [CARRADA Dataset: Camera and Automotive Radar with Range-Angle-Doppler Annotations](https://mlsanity.com/paper/2005.01456/) (2020)
- [T-FFTRadNet: Object Detection with Swin Vision Transformers from Raw ADC Radar Signals](https://mlsanity.com/paper/2303.16940/) (2023)
- [High Resolution Point Clouds from mmWave Radar](https://mlsanity.com/paper/2206.09273/) (2022)
- [Deep Radar Detector](https://mlsanity.com/paper/1906.12187/) (2019)
- [ColoRadar: The Direct 3D Millimeter Wave Radar Dataset](https://mlsanity.com/paper/2103.04510/) (2021)

## Abstract (arXiv)

In the context of firefighting and rescue operations, a cross-modal semantic segmentation model based on a single-chip millimeter-wave (mmWave) radar for indoor environmental perception is proposed and discussed. To efficiently obtain high-quality labels, an automatic label generation method utilizing LiDAR point clouds and occupancy grid maps is introduced. The proposed segmentation model is based on U-Net. A spatial attention module is incorporated, which enhanced the performance of the mode. The results demonstrate that cross-modal semantic segmentation provides a more intuitive and accurate representation of indoor environments. Unlike traditional methods, the model's segmentation performance is minimally affected by azimuth. Although performance declines with increasing distance, this can be mitigated by a well-designed model. Additionally, it was found that using raw ADC data as input is ineffective; compared to RA tensors, RD tensors are more suitable for the proposed model.