mlsanityData to 7 Oct 2026

FlowCam: Training Generalizable 3D Radiance Fields without Camera Poses via Pixel-Aligned Scene Flow

arXiv 2306.00180 ↗First posted 31 May 2023cs.CV cs.AI cs.GRcited by 38

Not on the map. Why?

Abstract (arXiv)

Reconstruction of 3D neural fields from posed images has emerged as a promising method for self-supervised representation learning. The key challenge preventing the deployment of these 3D scene learners on large-scale video data is their dependence on precise camera poses from structure-from-motion, which is prohibitively expensive to run at scale. We propose a method that jointly reconstructs camera poses and 3D neural scene representations online and in a single forward pass. We estimate poses by first lifting frame-to-frame optical flow to 3D scene flow via differentiable rendering, preserving locality and shift-equivariance of the image processing backbone. SE(3) camera pose estimation is then performed via a weighted least-squares fit to the scene flow field. This formulation enables us to jointly supervise pose estimation and a generalizable neural scene representation via re-rendering the input video, and thus, train end-to-end and fully self-supervised on real-world video datasets. We demonstrate that our method performs robustly on diverse, real-world video, notably on sequences traditionally challenging to optimization-based pose estimation techniques.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 55 of its 67 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

55 of 67 references.

References · 55

  1. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  4. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  7. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  8. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  9. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  10. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  11. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
38 more
  1. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  2. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  3. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  4. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  5. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  6. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  7. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  8. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  9. 2021Neural Fields in Visual Computing and Beyondcited by 539
  10. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  11. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  12. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 381
  13. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  14. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  15. 2020State of the Art on Neural Renderingcited by 276
  16. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  17. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
  18. 2020Deep Global Registrationcited by 228
  19. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  20. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  21. 2023Robust Dynamic Radiance Fieldscited by 193
  22. 2021Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenescited by 190
  23. 2020Robust Consistent Video Depth Estimationcited by 188
  24. 2022NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fieldscited by 188
  25. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  26. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  27. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  28. 2022Generalizable Patch-Based Neural Renderingcited by 121
  29. 2022Vision Transformer for NeRF-Based View Synthesis from a Single Input Imagecited by 104
  30. 2023Learning to Render Novel Views from Wide-Baseline Stereo Pairscited by 103
  31. 2022Orbeez-SLAM: A Real-time Monocular Visual SLAM with ORB Features and NeRF-realized Mappingcited by 79
  32. 2022Perspective Fields for Single Image Camera Calibrationcited by 75
  33. 2022ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformerscited by 70
  34. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  35. 2023DBARF: Deep Bundle-Adjusting Generalizable Neural Radiance Fieldscited by 60
  36. 2018RenderNet: A deep convolutional network for differentiable rendering from 3D shapescited by 42
  37. 2022RUST: Latent Neural Scene Representations from Unposed Imagerycited by 41
  38. 2018Learning Spatial Common Sense with Geometry-Aware Recurrent Networkscited by 35

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 10; 2025: 16; 2026: 2.

38 papers.

Sort:

By area:

  1. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  2. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  3. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  4. 2023COLMAP-Free 3D Gaussian Splattingcited by 193
  5. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 176
  6. 2024Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairscited by 171
  7. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  8. 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
  9. 2024PF3plat: Pose-Free Feed-Forward 3D Gaussian Splattingcited by 60
  10. 2024SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splattingcited by 40
  11. 2025No Pose at All: Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Viewscited by 38
  12. 2023Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairscited by 37
26 more
  1. 2023LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFscited by 33
  2. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  3. 2024GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-timecited by 20
  4. 2023UpFusion: Novel View Diffusion from Unposed Sparse View Observationscited by 14
  5. 2025True Self-Supervised Novel View Synthesis is Transferablecited by 13
  6. 2026TokenSplat: Token-aligned 3D Gaussian Splatting for Feed-forward Pose-free Reconstructioncited by 13
  7. 2023Cinematic Behavior Transfer via NeRF-based Differentiable Filmingcited by 12
  8. 2025RegGS: Unposed Sparse Views Gaussian Splatting with 3DGS Registrationcited by 12
  9. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  10. 2024UniCal: Unified Neural Sensor Calibrationcited by 8
  11. 2024Correspondence-Guided SfM-Free 3D Gaussian Splatting for NVScited by 7
  12. 2025SeqVLM: Proposal-Guided Multi-View Sequences Reasoning via VLM for Zero-Shot 3D Visual Groundingcited by 7
  13. 2025E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Modelscited by 6
  14. 2025SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Viewscited by 5
  15. 2023AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimizationcited by 4
  16. 2023Pose-Free Generalizable Rendering Transformercited by 4
  17. 2025SaLon3R: Structure-aware Long-term Generalizable 3D Reconstruction from Unposed Imagescited by 4
  18. 2025Flow-NeRF: Joint Learning of Geometry, Poses, and Dense Flow within Unified Neural Representationscited by 3
  19. 2025Recollection from Pensieve: Novel View Synthesis via Learning from Uncalibrated Videoscited by 3
  20. 2025Stylos: Multi-View 3D Stylization with Single-Forward Gaussian Splattingcited by 3
  21. 2024ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimizationcited by 1
  22. 2024DynSUP: Dynamic Gaussian Splatting from An Unposed Image Paircited by 1
  23. 2025Coca-Splat: Collaborative Optimization for Camera Parameters and 3D Gaussianscited by 1
  24. 2025Pose-free 3D Gaussian splatting via shape-ray estimationcited by 1
  25. 2025SIRE: SE(3) Intrinsic Rigidity Embeddingscited by 0
  26. 2026RayDer: Scalable Self-Supervised Novel View Synthesis from Real-World Videocited by 0