# CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstruction

arXiv: https://arxiv.org/abs/2304.14633 · HTML: https://mlsanity.com/paper/2304.14633/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 44 references that are in the map.
- First posted on arXiv 28 Apr 2023.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Point-Based Multi-View Stereo Network](https://mlsanity.com/paper/1908.04422/) (2019)
- [Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries](https://mlsanity.com/paper/1803.08673/) (2018)
- [DPSNet: End-to-end Deep Plane Sweep Stereo](https://mlsanity.com/paper/1905.00538/) (2019)
- [TransformerFusion: Monocular RGB Scene Reconstruction using Transformers](https://mlsanity.com/paper/2107.02191/) (2021)
- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion](https://mlsanity.com/paper/2012.02177/) (2020)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [MVDepthNet: Real-time Multiview Depth Estimation Neural Network](https://mlsanity.com/paper/1807.08563/) (2018)
- [MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera](https://mlsanity.com/paper/2011.11814/) (2020)
- [Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks](https://mlsanity.com/paper/2011.13118/) (2020)
- [CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2111.10502/) (2021)
- [RoutedFusion: Learning Real-time Depth Map Fusion](https://mlsanity.com/paper/2001.04388/) (2020)
- [Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depth](https://mlsanity.com/paper/2203.15174/) (2022)
- [Multi-View Stereo by Temporal Nonparametric Fusion](https://mlsanity.com/paper/1904.06397/) (2019)
- [NeuralFusion: Online Depth Fusion in Latent Space](https://mlsanity.com/paper/2011.14791/) (2020)
- [VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion](https://mlsanity.com/paper/2112.00236/) (2021)
- [Multimodal Semi-Supervised Learning for 3D Objects](https://mlsanity.com/paper/2110.11601/) (2021)
- [3DVNet: Multi-View Depth Prediction and Volumetric Refinement](https://mlsanity.com/paper/2112.00202/) (2021)
- [FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstruction](https://mlsanity.com/paper/2304.01480/) (2023)
- [Real-Time Dense 3D Mapping of Underwater Environments](https://mlsanity.com/paper/2304.02704/) (2023)
- [Class-Level Confidence Based 3D Semi-Supervised Learning](https://mlsanity.com/paper/2210.10138/) (2022)

## Built on by (most cited first, 16 of 16)

- [GaussianPro: 3D Gaussian Splatting with Progressive Propagation](https://mlsanity.com/paper/2402.14650/) (2024)
- [Fully Sparse 3D Occupancy Prediction](https://mlsanity.com/paper/2312.17118/) (2023)
- [Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation Models](https://mlsanity.com/paper/2305.08776/) (2023)
- [UC-NeRF: Neural Radiance Field for Under-Calibrated Multi-view Cameras in Autonomous Driving](https://mlsanity.com/paper/2311.16945/) (2023)
- [Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis](https://mlsanity.com/paper/2410.22817/) (2024)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [NC-SDF: Enhancing Indoor Scene Reconstruction Using Neural SDFs with View-Dependent Normal Compensation](https://mlsanity.com/paper/2405.00340/) (2024)
- [Ray-Distance Volume Rendering for Neural Scene Reconstruction](https://mlsanity.com/paper/2408.15524/) (2024)
- [SAM-Guided Masked Token Prediction for 3D Scene Understanding](https://mlsanity.com/paper/2410.12158/) (2024)
- [Depth-Consistent 3D Gaussian Splatting via Physical Defocus Modeling and Multi-View Geometric Supervision](https://mlsanity.com/paper/2511.10316/) (2025)
- [CityGuard: Graph-Aware Private Descriptors for Bias-Resilient Identity Search Across Urban Cameras](https://mlsanity.com/paper/2602.18047/) (2026)
- [PointRecon: Online Point-based 3D Reconstruction via Ray-based 2D-3D Matching](https://mlsanity.com/paper/2410.23245/) (2024)
- [Image-Plane Geometric Decoding for View-Invariant Indoor Scene Reconstruction](https://mlsanity.com/paper/2509.25744/) (2025)
- [UniFField: A Generalizable Unified Neural Feature Field for Visual, Semantic, and Spatial Uncertainties in Any Scene](https://mlsanity.com/paper/2510.06754/) (2025)
- [Deep learning Based Correction Algorithms for 3D Medical Reconstruction in Computed Tomography and Macroscopic Imaging](https://mlsanity.com/paper/2602.00220/) (2026)
- [VFM-Recon: Unlocking Cross-Domain Scene-Level Neural Reconstruction with Scale-Aligned Foundation Priors](https://mlsanity.com/paper/2603.12657/) (2026)

## Abstract (arXiv)

Recent advances in neural reconstruction using posed image sequences have made remarkable progress. However, due to the lack of depth information, existing volumetric-based techniques simply duplicate 2D image features of the object surface along the entire camera ray. We contend this duplication introduces noise in empty and occluded spaces, posing challenges for producing high-quality 3D geometry. Drawing inspiration from traditional multi-view stereo methods, we propose an end-to-end 3D neural reconstruction framework CVRecon, designed to exploit the rich geometric embedding in the cost volumes to facilitate 3D geometric feature learning. Furthermore, we present Ray-contextual Compensated Cost Volume (RCCV), a novel 3D geometric feature representation that encodes view-dependent information with improved integrity and robustness. Through comprehensive experiments, we demonstrate that our approach significantly improves the reconstruction quality in various metrics and recovers clear fine details of the 3D geometries. Our extensive ablation studies provide insights into the development of effective 3D geometric feature learning schemes. Project page: https://cvrecon.ziyue.cool/