mlsanityData to 7 Oct 2026

Structure-Aware NeRF without Posed Camera via Epipolar Constraint

arXiv 2210.00183 ↗First posted 1 Oct 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

The neural radiance field (NeRF) for realistic novel view synthesis requires camera poses to be pre-acquired by a structure-from-motion (SfM) approach. This two-stage strategy is not convenient to use and degrades the performance because the error in the pose extraction can propagate to the view synthesis. We integrate the pose extraction and view synthesis into a single end-to-end procedure so they can benefit from each other. For training NeRF models, only RGB images are given, without pre-known camera poses. The camera poses are obtained by the epipolar constraint in which the identical feature in different views has the same world coordinates transformed from the local camera coordinates according to the extracted poses. The epipolar constraint is jointly optimized with pixel color constraint. The poses are represented by a CNN-based deep network, whose input is the related frames. This joint optimization enables NeRF to be aware of the scene's structure that has an improved generalization performance. Extensive experiments on a variety of scenes demonstrate the effectiveness of the proposed approach. Code is available at https://github.com/XTU-PR-LAB/SaNerf.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 1 Oct 2022.

Builds on

44 of 63 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  8. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  9. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  10. 2020Neural Sparse Voxel Fieldscited by 959
  11. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  12. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
32 more
  1. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  2. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  3. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  4. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  5. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  6. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  7. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  8. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  9. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  10. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  11. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  12. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  13. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  14. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  15. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 381
  16. 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 301
  17. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  18. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  19. 2020AutoInt: Automatic Integration for Fast Neural Volume Renderingcited by 204
  20. 2021NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereocited by 191
  21. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  22. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  23. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  24. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  25. 2020Stable View Synthesiscited by 165
  26. 2021NeRF--: Neural Radiance Fields Without Known Camera Parameterscited by 155
  27. 2021GNeRF: GAN-based Neural Radiance Field without Posed Cameracited by 153
  28. 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
  29. 2019Learning to Infer Implicit Surfaces without 3D Supervisioncited by 103
  30. 2019KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objectscited by 75
  31. 2020SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Imagescited by 68
  32. 2017Deformable Shape Completion with Graph Convolutional Autoencoderscited by 50

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 1; 2025: 1.

6 papers.

  1. 2023LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFscited by 33
  2. 2024TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature Trackscited by 12
  3. 20253R-GS: Best Practice in Optimizing Camera Poses Along with 3DGScited by 10
  4. 2023NaviNeRF: NeRF-based 3D Representation Disentanglement by Latent Semantic Navigationcited by 8
  5. 2023DroNeRF: Real-time Multi-agent Drone Pose Optimization for Computing Neural Radiance Fieldscited by 4
  6. 2023MELON: NeRF with Unposed Images in SO(3)cited by 4