# Radar Spectra-Language Model for Automotive Scene Parsing

arXiv: https://arxiv.org/abs/2406.02158 · HTML: https://mlsanity.com/paper/2406.02158/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [LiT: Zero-Shot Transfer with Locked-image text Tuning](https://mlsanity.com/paper/2111.07991/) (2021)
- [NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario](https://mlsanity.com/paper/2305.14836/) (2023)
- [LLM4Drive: A Survey of Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.01043/) (2023)
- [K-Radar: 4D Radar Object Detection for Autonomous Driving in Various Weather Conditions](https://mlsanity.com/paper/2206.08171/) (2022)
- [DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment](https://mlsanity.com/paper/2304.04514/) (2023)
- [Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving](https://mlsanity.com/paper/2310.02251/) (2023)
- [Raw High-Definition Radar for Multi-Task Learning](https://mlsanity.com/paper/2112.10646/) (2021)
- [RADDet: Range-Azimuth-Doppler based Radar Object Detection for Dynamic Road Users](https://mlsanity.com/paper/2105.00363/) (2021)
- [2D Car Detection in Radar Data with PointNets](https://mlsanity.com/paper/1904.08414/) (2019)
- [Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Driving](https://mlsanity.com/paper/2309.14491/) (2023)
- [Improved Orientation Estimation and Detection with Hybrid Object Detection Networks for Automotive Radar](https://mlsanity.com/paper/2205.02111/) (2022)
- [LidarCLIP or: How I Learned to Talk to Point Clouds](https://mlsanity.com/paper/2212.06858/) (2022)
- [RadarDistill: Boosting Radar-based Object Detection Performance via Knowledge Distillation from LiDAR Features](https://mlsanity.com/paper/2403.05061/) (2024)
- [T-FFTRadNet: Object Detection with Swin Vision Transformers from Raw ADC Radar Signals](https://mlsanity.com/paper/2303.16940/) (2023)
- [DeepHybrid: Deep Learning on Automotive Radar Spectra and Reflections for Object Classification](https://mlsanity.com/paper/2202.08519/) (2022)
- [Zelda: Video Analytics using Vision-Language Models](https://mlsanity.com/paper/2305.03785/) (2023)

## Built on by (most cited first, 6 of 6)

- [Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systems](https://mlsanity.com/paper/2512.24385/) (2025)
- [RLM: A Vision-Language Model Approach for Radar Scene Understanding](https://mlsanity.com/paper/2511.21105/) (2025)
- [Weather-Robust Scene Semantics with Vision-Aligned 4D Radar](https://mlsanity.com/paper/2605.07367/) (2026)
- [WiFi2Cap: Semantic Action Captioning from Wi-Fi CSI via Limb-Level Semantic Alignment](https://mlsanity.com/paper/2603.22690/) (2026)
- [RAVEN: Radar Adaptive Vision Encoders for Efficient Chirp-wise Object Detection and Segmentation](https://mlsanity.com/paper/2604.04490/) (2026)
- [Radar4D-VLM: Proposal-Grounded Temporal 4D Radar Reasoning Across Frozen Language Models](https://mlsanity.com/paper/2608.04130/) (2026)

## Abstract (arXiv)

Radar sensors are low cost, long-range, and weather-resilient. Therefore, they are widely used for driver assistance functions, and are expected to be crucial for the success of autonomous driving in the future. In many perception tasks only pre-processed radar point clouds are considered. In contrast, radar spectra are a raw form of radar measurements and contain more information than radar point clouds. However, radar spectra are rather difficult to interpret. In this work, we aim to explore the semantic information contained in spectra in the context of automated driving, thereby moving towards better interpretability of radar spectra. To this end, we create a radar spectra-language model, allowing us to query radar spectra measurements for the presence of scene elements using free text. We overcome the scarcity of radar spectra data by matching the embedding space of an existing vision-language model. Finally, we explore the benefit of the learned representation for scene retrieval using radar spectra only, and obtain improvements in free space segmentation and object detection merely by injecting the spectra embedding into a baseline model.