# Selection, Not Fusion: Radar-Modulated State Space Models for Radar-Camera Depth Estimation

arXiv: https://arxiv.org/abs/2605.11840 · HTML: https://mlsanity.com/paper/2605.11840/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Point clouds, detection and remote sensing, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 12 May 2026.

## Placed in

- State-space sequence models (S4, Mamba) / [Point clouds, detection and remote sensing](https://mlsanity.com/ssm/point-clouds-detection-and-remote-sensing/)

## Builds on (most cited first, 23 of 23)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Learning Combinatorial Embedding Networks for Deep Graph Matching](https://mlsanity.com/paper/1904.00597/) (2019)
- [RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection](https://mlsanity.com/paper/2403.16440/) (2024)
- [FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba](https://mlsanity.com/paper/2404.09498/) (2024)
- [Radar-Camera Pixel Depth Association for Depth Completion](https://mlsanity.com/paper/2106.02778/) (2021)
- [Depth Estimation from Monocular Images and Sparse Radar Data](https://mlsanity.com/paper/2010.00058/) (2020)
- [R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes](https://mlsanity.com/paper/2108.04814/) (2021)
- [MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion](https://mlsanity.com/paper/2404.08406/) (2024)
- [Depth Estimation from Monocular Images and Sparse radar using Deep Ordinal Regression Network](https://mlsanity.com/paper/2107.07596/) (2021)
- [4D Millimeter-Wave Radar in Autonomous Driving: A Survey](https://mlsanity.com/paper/2306.04242/) (2023)
- [RadarCam-Depth: Radar-Camera Fusion for Depth Estimation with Learned Metric Scale](https://mlsanity.com/paper/2401.04325/) (2024)
- [RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion](https://mlsanity.com/paper/2412.12725/) (2024)
- [Sparse Beats Dense: Rethinking Supervision in Radar-Camera Depth Completion](https://mlsanity.com/paper/2312.00844/) (2023)
- [CaFNet: A Confidence-Driven Framework for Radar Camera Depth Estimation](https://mlsanity.com/paper/2407.00697/) (2024)
- [Lidar-level localization with radar? The CFEAR approach to accurate, fast and robust large-scale radar odometry in diverse environments](https://mlsanity.com/paper/2211.02445/) (2022)
- [TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion](https://mlsanity.com/paper/2504.11773/) (2025)
- [MambaDepth: Enhancing Long-range Dependency for Self-Supervised Fine-Structured Monocular Depth Estimation](https://mlsanity.com/paper/2406.04532/) (2024)

## Built on by (most cited first, 1 of 1)

- [RbFT-Net: Rectify-Before-Fuse Temporal Radar Anchors for 4D Radar-Camera Depth Completion](https://mlsanity.com/paper/2608.13102/) (2026)

## Abstract (arXiv)

Radar camera depth estimation turns a very sparse but metric radar signal into a dense depth map. Existing methods combine radar and image features outside the sequence operator of the backbone, through concatenation, confidence gating, sparse supervision, or graph based extraction. Mamba variants for multimodal fusion also leave the selection mechanism itself unimodal. We argue that selection is the right place for radar to enter. We introduce Radar-Modulated Selection (RMS), which injects radar inside the selective scan of Mamba. Radar adds learned perturbations, initialised at zero, to the step size Delta and the readout C. The input projection B and the state matrix A never see radar. At initialisation the block equals a pretrained image Mamba, so radar gains influence only where it improves accuracy. The construction also couples the two modalities at linear cost and falls back to the image backbone when radar is missing. We deploy RMS in a Multi-View Scan Pyramid (MVSP) that matches the fusion operator to the reach of radar at each scale. On nuScenes, SemoDepth reduces MAE by 34.0%, 29.9%, and 29.9% over the previous best within 50, 70, and 80 m. It also has the lowest single frame latency among methods with a public release (26.8 ms). A feature blend placed before the scan adds no accuracy on top of RMS, which supports the view that selection inside the scan can replace fusion outside it. When twice as many spurious radar returns as true ones are injected, the MAE@80 of SemoDepth rises by 1.4%, against 33.6% for a Mamba baseline that fuses radar before the scan.