mlsanityData to 7 Oct 2026

Combining HoloLens with Instant-NeRFs: Advanced Real-Time 3D Mobile Mapping

arXiv 2304.14301 ↗First posted 27 Apr 2023cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

This work represents a large step into modern ways of fast 3D reconstruction based on RGB camera images. Utilizing a Microsoft HoloLens 2 as a multisensor platform that includes an RGB camera and an inertial measurement unit for SLAM-based camera-pose determination, we train a Neural Radiance Field (NeRF) as a neural scene representation in real-time with the acquired data from the HoloLens. The HoloLens is connected via Wifi to a high-performance PC that is responsible for the training and 3D reconstruction. After the data stream ends, the training is stopped and the 3D reconstruction is initiated, which extracts a point cloud of the scene. With our specialized inference algorithm, five million scene points can be extracted within 1 second. In addition, the point cloud also includes radiometry per point. Our method of 3D reconstruction outperforms grid point sampling with NeRFs by multiple orders of magnitude and can be regarded as a complete real-time 3D reconstruction method in a mobile mapping setup.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

22 of 41 references.

References · 22

  1. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  4. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  5. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  6. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  9. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  10. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  11. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  12. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
10 more
  1. 2021Urban Radiance Fieldscited by 263
  2. 2021Self-Calibrating Neural Radiance Fieldscited by 204
  3. 2021Deblur-NeRF: Neural Radiance Fields from Blurry Imagescited by 179
  4. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  5. 2023RobustNeRF: Ignoring Distractors with Robust Lossescited by 101
  6. 2021T"oRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesiscited by 88
  7. 2020Neural Volume Rendering: NeRF And Beyondcited by 44
  8. 2022HoloLens 2 Sensor Streamingcited by 11
  9. 2023A Comparative Neural Radiance Field (NeRF) 3D Analysis of Camera Poses from HoloLens Trajectories and Structure from Motioncited by 3
  10. 2022Corrosion Detection for Industrial Objects: From Multi-Sensor System to 5D Feature Spacecited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023An Outlook into the Future of Egocentric Visioncited by 84