# FlowCam: Training Generalizable 3D Radiance Fields without Camera Poses via Pixel-Aligned Scene Flow

arXiv: https://arxiv.org/abs/2306.00180 · HTML: https://mlsanity.com/paper/2306.00180/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 67 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 30 of 55)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [Generative Novel View Synthesis with 3D-Aware Diffusion Models](https://mlsanity.com/paper/2304.02602/) (2023)
- [NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior](https://mlsanity.com/paper/2212.07388/) (2022)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)

## Built on by (most cited first, 30 of 39)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images](https://mlsanity.com/paper/2410.24207/) (2024)
- [COLMAP-Free 3D Gaussian Splatting](https://mlsanity.com/paper/2312.07504/) (2023)
- [FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views](https://mlsanity.com/paper/2502.12138/) (2025)
- [Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs](https://mlsanity.com/paper/2408.13912/) (2024)
- [PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction](https://mlsanity.com/paper/2311.12024/) (2023)
- [PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting](https://mlsanity.com/paper/2410.22128/) (2024)
- [FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent](https://mlsanity.com/paper/2404.15259/) (2024)
- [SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting](https://mlsanity.com/paper/2411.17190/) (2024)
- [No Pose at All: Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views](https://mlsanity.com/paper/2508.01171/) (2025)
- [Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs](https://mlsanity.com/paper/2312.07246/) (2023)
- [LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFs](https://mlsanity.com/paper/2306.05410/) (2023)
- [Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey](https://mlsanity.com/paper/2507.14501/) (2025)
- [GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-time](https://mlsanity.com/paper/2403.10147/) (2024)
- [True Self-Supervised Novel View Synthesis is Transferable](https://mlsanity.com/paper/2510.13063/) (2025)
- [UpFusion: Novel View Diffusion from Unposed Sparse View Observations](https://mlsanity.com/paper/2312.06661/) (2023)
- [TokenSplat: Token-aligned 3D Gaussian Splatting for Feed-forward Pose-free Reconstruction](https://mlsanity.com/paper/2603.00697/) (2026)
- [RegGS: Unposed Sparse Views Gaussian Splatting with 3DGS Registration](https://mlsanity.com/paper/2507.08136/) (2025)
- [Cinematic Behavior Transfer via NeRF-based Differentiable Filming](https://mlsanity.com/paper/2311.17754/) (2023)
- [Can Video Diffusion Model Reconstruct 4D Geometry?](https://mlsanity.com/paper/2503.21082/) (2025)
- [Correspondence-Guided SfM-Free 3D Gaussian Splatting for NVS](https://mlsanity.com/paper/2408.08723/) (2024)
- [UniCal: Unified Neural Sensor Calibration](https://mlsanity.com/paper/2409.18953/) (2024)
- [E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models](https://mlsanity.com/paper/2506.01933/) (2025)
- [SeqVLM: Proposal-Guided Multi-View Sequences Reasoning via VLM for Zero-Shot 3D Visual Grounding](https://mlsanity.com/paper/2508.20758/) (2025)
- [SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views](https://mlsanity.com/paper/2509.17246/) (2025)
- [Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective](https://mlsanity.com/paper/2604.14025/) (2026)
- [AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization](https://mlsanity.com/paper/2308.10001/) (2023)
- [Pose-Free Generalizable Rendering Transformer](https://mlsanity.com/paper/2310.03704/) (2023)
- [SaLon3R: Structure-aware Long-term Generalizable 3D Reconstruction from Unposed Images](https://mlsanity.com/paper/2510.15072/) (2025)

## Abstract (arXiv)

Reconstruction of 3D neural fields from posed images has emerged as a promising method for self-supervised representation learning. The key challenge preventing the deployment of these 3D scene learners on large-scale video data is their dependence on precise camera poses from structure-from-motion, which is prohibitively expensive to run at scale. We propose a method that jointly reconstructs camera poses and 3D neural scene representations online and in a single forward pass. We estimate poses by first lifting frame-to-frame optical flow to 3D scene flow via differentiable rendering, preserving locality and shift-equivariance of the image processing backbone. SE(3) camera pose estimation is then performed via a weighted least-squares fit to the scene flow field. This formulation enables us to jointly supervise pose estimation and a generalizable neural scene representation via re-rendering the input video, and thus, train end-to-end and fully self-supervised on real-world video datasets. We demonstrate that our method performs robustly on diverse, real-world video, notably on sequences traditionally challenging to optimization-based pose estimation techniques.