# mmIR: Frequency-Space Inverse Rendering for 3D Millimeter-Wave Radar ADC Synthesis

arXiv: https://arxiv.org/abs/2608.28913 · HTML: https://mlsanity.com/paper/2608.28913/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 56 references that are in the map.
- First posted on arXiv 28 Aug 2026.

## Builds on (most cited first, 20 of 20)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset](https://mlsanity.com/paper/1909.01300/) (2019)
- [Differentiable Rendering: A Survey](https://mlsanity.com/paper/2006.12057/) (2020)
- [IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images](https://mlsanity.com/paper/2204.02232/) (2022)
- [Sionna RT: Differentiable Ray Tracing for Radio Propagation Modeling](https://mlsanity.com/paper/2303.11103/) (2023)
- [ColoRadar: The Direct 3D Millimeter Wave Radar Dataset](https://mlsanity.com/paper/2103.04510/) (2021)
- [DART: Implicit Doppler Tomography for Radar Novel View Synthesis](https://mlsanity.com/paper/2403.03896/) (2024)
- [Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar](https://mlsanity.com/paper/2405.04662/) (2024)
- [RadarSplat: Radar Gaussian Splatting for High-Fidelity Data Synthesis and 3D Reconstruction of Autonomous Driving Scenes](https://mlsanity.com/paper/2506.01379/) (2025)
- [Experiments with mmWave Automotive Radar Test-bed](https://mlsanity.com/paper/1912.12566/) (2019)
- [SAR-NeRF: Neural Radiance Fields for Synthetic Aperture Radar Multi-View Representation](https://mlsanity.com/paper/2307.05087/) (2023)
- [DiffSound: Differentiable Modal Sound Rendering and Inverse Rendering for Diverse Inference Tasks](https://mlsanity.com/paper/2409.13486/) (2024)
- [NeuRadar: Neural Radiance Fields for Automotive Radar Point Clouds](https://mlsanity.com/paper/2504.00859/) (2025)
- [SpINR: Neural Volumetric Reconstruction for FMCW Radars](https://mlsanity.com/paper/2503.23313/) (2025)
- [InverTwin: Solving Inverse Problems via Differentiable Radio Frequency Digital Twin](https://mlsanity.com/paper/2508.14204/) (2025)
- [MITO: A Millimeter-Wave Dataset and Simulator for Non-Line-of-Sight Perception](https://mlsanity.com/paper/2502.10259/) (2025)
- [RF4D:Neural Radar Fields for Novel View Synthesis in Outdoor Dynamic Scenes](https://mlsanity.com/paper/2505.20967/) (2025)
- [Learning Surface Scattering Parameters From SAR Images Using Differentiable Ray Tracing](https://mlsanity.com/paper/2401.01175/) (2024)
- [MAROON: A Dataset for the Joint Characterization of Near-Field High-Resolution Radio-Frequency and Optical Depth Imaging Techniques](https://mlsanity.com/paper/2411.00527/) (2024)

## Built on by (most cited first, 1 of 1)

- [DyRAD: Radar Novel View Synthesis for Dynamic Driving Scenes](https://mlsanity.com/paper/2609.39841/) (2026)

## Abstract (arXiv)

High-resolution 3D radar data is scarce. Commodity mmWave sensors use small antenna arrays that limit angular resolution to several degrees, and existing datasets provide only 2D range-azimuth maps or sparse point clouds rather than raw analog-to-digital converter (ADC) signals. Hardware scaling is expensive, synthetic-aperture scanning is impractical at fleet scale, and learned synthesis methods are bottlenecked by the very data shortage they aim to address. We present mmIR, an open-source differentiable frequency-modulated continuous-wave (FMCW) radar inverse renderer that fits a physics-based forward model to real captures and re-renders from dense virtual apertures to synthesize high-resolution 3D radar data. Because radar resolution is too coarse to recover geometry directly, mmIR performs LiDAR-assisted inverse rendering: using LiDAR-derived meshes as a geometric scaffold, mmIR optimizes per-vertex International Telecommunication Union (ITU) physics materials, vertex normals, and antenna beam patterns through end-to-end automatic differentiation of a phase-coherent multiple-input multiple-output (MIMO) forward model with multi-bounce propagation, polarization, and free-space diffraction. On seven outdoor and six indoor ColoRadar scenes, mmIR achieves 0.914 mean Pearson correlation on range-azimuth maps versus 0.307 for Sionna-RT. Scenes trained on a cascaded imaging radar transfer to a co-located single-chip radar without re-training (0.554 correlation), and dense virtual arrays (100x100 elements) produce single-frame 3D occupancy validated against LiDAR. Project page: https://mmwave-inverse-rendering.github.io/