# SOAC: Spatio-Temporal Overlap-Aware Multi-Sensor Calibration using Neural Radiance Fields arXiv: https://arxiv.org/abs/2311.15803 ยท HTML: https://mlsanity.com/paper/2311.15803/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Builds on 29 of its 51 references that are in the map. - First posted on arXiv 27 Nov 2023. ## Builds on (most cited first, 29 of 29) - [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019) - [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022) - [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021) - [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021) - [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021) - [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015) - [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021) - [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021) - [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021) - [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020) - [NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior](https://mlsanity.com/paper/2212.07388/) (2022) - [Self-Calibrating Neural Radiance Fields](https://mlsanity.com/paper/2108.13826/) (2021) - [SPARF: Neural Radiance Fields from Sparse and Noisy Poses](https://mlsanity.com/paper/2211.11738/) (2022) - [PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving](https://mlsanity.com/paper/2112.12610/) (2021) - [NeRF--: Neural Radiance Fields Without Known Camera Parameters](https://mlsanity.com/paper/2102.07064/) (2021) - [GNeRF: GAN-based Neural Radiance Field without Posed Camera](https://mlsanity.com/paper/2103.15606/) (2021) - [KISS-ICP: In Defense of Point-to-Point ICP -- Simple, Accurate, and Robust Registration If Done the Right Way](https://mlsanity.com/paper/2209.15397/) (2022) - [RegNet: Multimodal Sensor Registration Using Deep Neural Networks](https://mlsanity.com/paper/1707.03167/) (2017) - [Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields](https://mlsanity.com/paper/2209.09050/) (2022) - [Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments](https://mlsanity.com/paper/2103.01627/) (2021) - [CROSSFIRE: Camera Relocalization On Self-Supervised Features from an Implicit Representation](https://mlsanity.com/paper/2303.04869/) (2023) - [nerf2nerf: Pairwise Registration of Neural Radiance Fields](https://mlsanity.com/paper/2211.01600/) (2022) - [MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration](https://mlsanity.com/paper/2303.03056/) (2023) - [Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups](https://mlsanity.com/paper/1705.04085/) (2017) - [DXQ-Net: Differentiable LiDAR-Camera Extrinsic Calibration Using Quality-aware Flow](https://mlsanity.com/paper/2203.09385/) (2022) - [Spatiotemporal Camera-LiDAR Calibration: A Targetless and Structureless Approach](https://mlsanity.com/paper/2001.06175/) (2020) - [INF: Implicit Neural Fusion for LiDAR and Camera](https://mlsanity.com/paper/2308.14414/) (2023) - [Pose-Free Neural Radiance Fields via Implicit Pose Regularization](https://mlsanity.com/paper/2308.15049/) (2023) - [SST-Calib: Simultaneous Spatial-Temporal Parameter Calibration between LIDAR and Camera](https://mlsanity.com/paper/2207.03704/) (2022) ## Built on by (most cited first, 13 of 13) - [Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives](https://mlsanity.com/paper/2402.02968/) (2024) - [RoDUS: Robust Decomposition of Static and Dynamic Elements in Urban Scenes](https://mlsanity.com/paper/2403.09419/) (2024) - [3DGS-Calib: 3D Gaussian Splatting for Multimodal SpatioTemporal Calibration](https://mlsanity.com/paper/2403.11577/) (2024) - [UniCal: Unified Neural Sensor Calibration](https://mlsanity.com/paper/2409.18953/) (2024) - [Refining Ground Truth Poses in Autonomous Driving Datasets via Neural Rendering](https://mlsanity.com/paper/2504.15776/) (2025) - [BEVCALIB: LiDAR-Camera Calibration via Geometry-Guided Bird's-Eye View Representations](https://mlsanity.com/paper/2506.02587/) (2025) - [UniQueR: Unified Query-based Feedforward 3D Reconstruction](https://mlsanity.com/paper/2603.22851/) (2026) - [Cal or No Cal? -- Real-Time Miscalibration Detection of LiDAR and Camera Sensors](https://mlsanity.com/paper/2504.01040/) (2025) - [RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields](https://mlsanity.com/paper/2605.26328/) (2026) - [Mind the Hitch: Dynamic Calibration and Articulated Perception for Autonomous Trucks](https://mlsanity.com/paper/2603.23711/) (2026) - [Native-Domain Cross-Attention for Camera-LiDAR Extrinsic Calibration Under Large Initial Perturbations](https://mlsanity.com/paper/2603.29414/) (2026) - [SpectralSplat: Appearance-Disentangled Feed-Forward Gaussian Splatting for Driving Scenes](https://mlsanity.com/paper/2604.03462/) (2026) - [Geometry-Preserving in 3D Gaussian Splatting for LiDAR-Camera Extrinsic Calibration](https://mlsanity.com/paper/2606.20103/) (2026) ## Abstract (arXiv) In rapidly-evolving domains such as autonomous driving, the use of multiple sensors with different modalities is crucial to ensure high operational precision and stability. To correctly exploit the provided information by each sensor in a single common frame, it is essential for these sensors to be accurately calibrated. In this paper, we leverage the ability of Neural Radiance Fields (NeRF) to represent different sensors modalities in a common volumetric representation to achieve robust and accurate spatio-temporal sensor calibration. By designing a partitioning approach based on the visible part of the scene for each sensor, we formulate the calibration problem using only the overlapping areas. This strategy results in a more robust and accurate calibration that is less prone to failure. We demonstrate that our approach works on outdoor urban scenes by validating it on multiple established driving datasets. Results show that our method is able to get better accuracy and robustness compared to existing methods.