SCOPE-4D: Endoscopic 4D Geometry Foundation Models
Not on the map. Why?
Abstract (arXiv)
Geometric understanding supports endoscopic navigation and robotic assistance, but learning reliable endoscopic geometry faces two challenges: scarce geometric annotations and ambiguity between camera motion and tissue deformation. We present SCOPE-4D, an endoscopic 4D geometry foundation model that jointly predicts camera parameters, dense geometry, and 3D tissue trajectories from monocular RGB video in a single forward pass. Our curation and annotation pipeline constructs SCOPE-5K, a collection of approximately 5,000 clips spanning real and synthetic gastrointestinal endoscopy and laparoscopy. The collection provides rich geometric supervision and includes newly collected phantom and real-colonoscopy evaluation sets. Geometric supervised fine-tuning on SCOPE-5K learns endoscopic priors that improve camera and depth estimation. Common--Residual Motion (CRM) further constrains local deformation relative to common tissue movement. Together with geometric supervision, CRM and trajectory supervision further improve camera and depth estimation over geometric fine-tuning alone while enabling dense 3D tissue tracking. Evaluations on public and newly collected benchmarks demonstrate strong in-domain and out-of-domain geometry, superior 3D tracking, and more stable long-sequence colon reconstruction. A blinded user study further supports the perceived reconstruction quality on real clinical video. Together, these results demonstrate the value of large-scale endoscopic supervision and motion constraints for joint geometry estimation and tissue tracking.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 47 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
40 of 47 references.
References · 40
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2023LightGlue: Local Feature Matching at Light Speedcited by 590
- 2025Continuous 3D Perception Model with Persistent Statecited by 548
- 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
- 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
- 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 141
- 2018A micro Lie theory for state estimation in roboticscited by 124
- 2022Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgerycited by 123
28 more
- 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequencescited by 115
- 2021Stereo Correspondence and Reconstruction of Endoscopic Data Challengecited by 103
- 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 87
- 2021Endo-Depth-and-Motion: Reconstruction and Tracking in Endoscopic Videos using Depth Networks and Photometric Constraintscited by 81
- 2022Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registrationcited by 80
- 2021Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescuecited by 72
- 2023EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videoscited by 69
- 2023MFT: Long-Term Tracking of Every Pixelcited by 62
- 2025Efficiently Reconstructing Dynamic Scenes One D4RT at a Timecited by 54
- 2023Learning How To Robustly Estimate Camera Pose in Endoscopic Videoscited by 50
- 2019Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methodscited by 47
- 2024EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Cameracited by 43
- 2025Any4D: Unified Feed-Forward Metric 4D Reconstructioncited by 35
- 2024EndoGaussian: Real-time Gaussian Splatting for Dynamic Endoscopic Scene Reconstructioncited by 22
- 20264RC: 4D Reconstruction via Conditional Querying Anytime and Anywherecited by 22
- 2023SimCol3D -- 3D Reconstruction during Colonoscopy Challengecited by 19
- 2025Endo3R: Unified Online Reconstruction from Dynamic Monocular Endoscopic Videocited by 16
- 2025C3VDv2 -- Colonoscopy 3D video dataset with enhanced realismcited by 15
- 2026LoMa: Local Feature Matching Revisitedcited by 11
- 2024Online 3D reconstruction and dense tracking in endoscopic videoscited by 10
- 2025Stereo Any Video: Temporally Consistent Stereo Matchingcited by 8
- 2026Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Roboticscited by 8
- 2022SuperPoint features in endoscopycited by 6
- 2025Endo-FASt3r: Endoscopic Foundation model Adaptation for Structure from motioncited by 6
- 2026SM4RT: Learning Structured Motion Geometry for 4D Reconstructioncited by 5
- 2026RealSynCol: a high-fidelity synthetic colon dataset for 3D reconstruction applicationscited by 2
- 2026NRGS-SLAM: Monocular Non-Rigid SLAM for Endoscopy via Deformation-Aware 3D Gaussian Splattingcited by 2
- 2025Learning to Efficiently Adapt Foundation Models for Self-Supervised Endoscopic 3D Scene Reconstruction from Any Camerascited by 1
Built on by
Not cited yet.