# Improving Robustness for Joint Optimization of Camera Poses and Decomposed Low-Rank Tensorial Radiance Fields

arXiv: https://arxiv.org/abs/2402.13252 · HTML: https://mlsanity.com/paper/2402.13252/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time](https://mlsanity.com/paper/2202.08614/) (2022)
- [Progressively Optimized Local Radiance Fields for Robust View Synthesis](https://mlsanity.com/paper/2303.13791/) (2023)
- [A Structured Dictionary Perspective on Implicit Neural Representations](https://mlsanity.com/paper/2112.01917/) (2021)
- [AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels](https://mlsanity.com/paper/2211.13206/) (2022)
- [Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedra](https://mlsanity.com/paper/2304.09987/) (2023)
- [Robust Camera Pose Refinement for Multi-Resolution Hash Encoding](https://mlsanity.com/paper/2302.01571/) (2023)
- [Multiscale Tensor Decomposition and Rendering Equation Encoding for View Synthesis](https://mlsanity.com/paper/2303.03808/) (2023)
- [TriVol: Point Cloud Rendering via Triple Volumes](https://mlsanity.com/paper/2303.16485/) (2023)

## Built on by (most cited first, 15 of 15)

- [LongSplat: Robust Unposed 3D Gaussian Splatting for Casual Long Videos](https://mlsanity.com/paper/2508.14041/) (2025)
- [AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360deg Unbounded Scene Inpainting](https://mlsanity.com/paper/2502.05176/) (2025)
- [FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors](https://mlsanity.com/paper/2410.16271/) (2024)
- [BoostMVSNeRFs: Boosting MVS-based NeRFs to Generalizable View Synthesis in Large-scale Scenes](https://mlsanity.com/paper/2407.15848/) (2024)
- [Graph-Guided Scene Reconstruction from Images with 3D Gaussian Splatting](https://mlsanity.com/paper/2502.17377/) (2025)
- [StealthAttack: Robust 3D Gaussian Splatting Poisoning via Density-Guided Illusions](https://mlsanity.com/paper/2510.02314/) (2025)
- [SG-NeRF: Neural Surface Reconstruction with Scene Graph Optimization](https://mlsanity.com/paper/2407.12667/) (2024)
- [Look Gauss, No Pose: Novel View Synthesis using Gaussian Splatting without Accurate Pose Initialization](https://mlsanity.com/paper/2410.08743/) (2024)
- [Voxify3D: Pixel Art Meets Volumetric Rendering](https://mlsanity.com/paper/2512.07834/) (2025)
- [Splannequin: Freezing Monocular Mannequin-Challenge Footage with Dual-Detection Splatting](https://mlsanity.com/paper/2512.05113/) (2025)
- [Robust SG-NeRF: Robust Scene Graph Aided Neural Surface Reconstruction](https://mlsanity.com/paper/2411.13620/) (2024)
- [KeyGS: A Keyframe-Centric Gaussian Splatting Method for Monocular Image Sequences](https://mlsanity.com/paper/2412.20767/) (2024)
- [Every Camera Effect, Every Time, All at Once: 4D Gaussian Ray Tracing for Physics-based Camera Effect Data Generation](https://mlsanity.com/paper/2509.10759/) (2025)
- [PCM-NeRF: Probabilistic Camera Modeling for Neural Radiance Fields under Pose Uncertainty](https://mlsanity.com/paper/2604.17831/) (2026)
- [Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction](https://mlsanity.com/paper/2609.04201/) (2026)

## Abstract (arXiv)

In this paper, we propose an algorithm that allows joint refinement of camera pose and scene geometry represented by decomposed low-rank tensor, using only 2D images as supervision. First, we conduct a pilot study based on a 1D signal and relate our findings to 3D scenarios, where the naive joint pose optimization on voxel-based NeRFs can easily lead to sub-optimal solutions. Moreover, based on the analysis of the frequency spectrum, we propose to apply convolutional Gaussian filters on 2D and 3D radiance fields for a coarse-to-fine training schedule that enables joint camera pose optimization. Leveraging the decomposition property in decomposed low-rank tensor, our method achieves an equivalent effect to brute-force 3D convolution with only incurring little computational overhead. To further improve the robustness and stability of joint optimization, we also propose techniques of smoothed 2D supervision, randomly scaled kernel parameters, and edge-guided loss mask. Extensive quantitative and qualitative evaluations demonstrate that our proposed framework achieves superior performance in novel view synthesis as well as rapid convergence for optimization.