# Learning Robust Multi-Scale Representation for Neural Radiance Fields from Unposed Images

arXiv: https://arxiv.org/abs/2311.04521 · HTML: https://mlsanity.com/paper/2311.04521/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 57 references that are in the map.
- First posted on arXiv 8 Nov 2023.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior](https://mlsanity.com/paper/2212.07388/) (2022)
- [Self-Calibrating Neural Radiance Fields](https://mlsanity.com/paper/2108.13826/) (2021)
- [NeRF--: Neural Radiance Fields Without Known Camera Parameters](https://mlsanity.com/paper/2102.07064/) (2021)
- [NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields](https://mlsanity.com/paper/2203.01913/) (2022)
- [ACORN: Adaptive Coordinate Networks for Neural Scene Representation](https://mlsanity.com/paper/2105.02788/) (2021)
- [Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields](https://mlsanity.com/paper/2209.08409/) (2022)
- [VA-DepthNet: A Variational Approach to Single Image Depth Prediction](https://mlsanity.com/paper/2302.06556/) (2023)
- [NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotations](https://mlsanity.com/paper/2306.09109/) (2023)
- [Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces](https://mlsanity.com/paper/2012.06777/) (2020)
- [NeuRoRA: Neural Robust Rotation Averaging](https://mlsanity.com/paper/1912.04485/) (2019)
- [Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2110.05594/) (2021)
- [Organic Priors in Non-Rigid Structure from Motion](https://mlsanity.com/paper/2207.06262/) (2022)
- [Robustifying the Multi-Scale Representation of Neural Radiance Fields](https://mlsanity.com/paper/2210.04233/) (2022)

## Built on by (most cited first, 7 of 7)

- [Stereo Risk: A Continuous Modeling Approach to Stereo Matching](https://mlsanity.com/paper/2407.03152/) (2024)
- [Reflections Unlock: Geometry-Aware Reflection Disentanglement in 3D Gaussian Splatting for Photorealistic Scenes Rendering](https://mlsanity.com/paper/2507.06103/) (2025)
- [OPONeRF: One-Point-One NeRF for Robust Neural Rendering](https://mlsanity.com/paper/2409.20043/) (2024)
- [Mobile Robotic Multi-View Photometric Stereo](https://mlsanity.com/paper/2502.10842/) (2025)
- [Time-Archival Camera Virtualization for Sports and Visual Performances](https://mlsanity.com/paper/2602.15181/) (2026)
- [Rethinking Dense Optical Flow without Test-Time Scaling](https://mlsanity.com/paper/2605.08000/) (2026)
- [3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark](https://mlsanity.com/paper/2605.12437/) (2026)

## Abstract (arXiv)

We introduce an improved solution to the neural image-based rendering problem in computer vision. Given a set of images taken from a freely moving camera at train time, the proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time. The key ideas presented in this paper are (i) Recovering accurate camera parameters via a robust pipeline from unposed day-to-day images is equally crucial in neural novel view synthesis problem; (ii) It is rather more practical to model object's content at different resolutions since dramatic camera motion is highly likely in day-to-day unposed images. To incorporate the key ideas, we leverage the fundamentals of scene rigidity, multi-scale neural scene representation, and single-image depth prediction. Concretely, the proposed approach makes the camera parameters as learnable in a neural fields-based modeling framework. By assuming per view depth prediction is given up to scale, we constrain the relative pose between successive frames. From the relative poses, absolute camera pose estimation is modeled via a graph-neural network-based multiple motion averaging within the multi-scale neural-fields network, leading to a single loss function. Optimizing the introduced loss function provides camera intrinsic, extrinsic, and image rendering from unposed images. We demonstrate, with examples, that for a unified framework to accurately model multiscale neural scene representation from day-to-day acquired unposed multi-view images, it is equally essential to have precise camera-pose estimates within the scene representation framework. Without considering robustness measures in the camera pose estimation pipeline, modeling for multi-scale aliasing artifacts can be counterproductive. We present extensive experiments on several benchmark datasets to demonstrate the suitability of our approach.