# Future Rendering neq Future Surface: A Benchmark and Dataset for Dynamic Surface Reconstruction Beyond the Observed Window

arXiv: https://arxiv.org/abs/2607.21471 · HTML: https://mlsanity.com/paper/2607.21471/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 28 references that are in the map.
- First posted on arXiv 23 Jul 2026.

## Builds on (most cited first, 27 of 27)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics](https://mlsanity.com/paper/2311.12198/) (2023)
- [Monocular Dynamic View Synthesis: A Reality Check](https://mlsanity.com/paper/2210.13445/) (2022)
- [Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians](https://mlsanity.com/paper/2403.09434/) (2024)
- [Croissant: A Metadata Format for ML-Ready Datasets](https://mlsanity.com/paper/2403.19546/) (2024)
- [GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation](https://mlsanity.com/paper/2406.14927/) (2024)
- [GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis](https://mlsanity.com/paper/2405.19745/) (2024)
- [Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Dynamic Scenes](https://mlsanity.com/paper/2404.12379/) (2024)
- [FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Video](https://mlsanity.com/paper/2503.04720/) (2025)
- [DynaSurfGS: Dynamic Surface Reconstruction with Planar-based Gaussian Splatting](https://mlsanity.com/paper/2408.13972/) (2024)
- [GauSTAR: Gaussian Surface Tracking and Reconstruction](https://mlsanity.com/paper/2501.10283/) (2025)
- [ODE-GS: Latent ODEs for Dynamic Scene Extrapolation with 3D Gaussian Splatting](https://mlsanity.com/paper/2506.05480/) (2025)
- [Space-time 2D Gaussian Splatting for Accurate Surface Reconstruction under Complex Dynamic Scenes](https://mlsanity.com/paper/2409.18852/) (2024)
- [ReconPhys: Reconstruct Appearance and Physical Attributes from Single Video](https://mlsanity.com/paper/2604.07882/) (2026)
- [4D Gaussian Splatting as a Learned Dynamical System](https://mlsanity.com/paper/2512.19648/) (2025)
- [Space-Time Forecasting of Dynamic Scenes with Motion-aware Gaussian Grouping](https://mlsanity.com/paper/2602.21668/) (2026)
- [GP-4DGS: Probabilistic 4D Gaussian Splatting from Monocular Video via Variational Gaussian Processes](https://mlsanity.com/paper/2604.02915/) (2026)
- [textit4DSurf: High-Fidelity Dynamic Scene Surface Reconstruction](https://mlsanity.com/paper/2603.28064/) (2026)
- [DySurface: Consistent 4D Surface Reconstruction via Bridging Explicit Gaussians and Implicit Functions](https://mlsanity.com/paper/2605.10360/) (2026)

## Abstract (arXiv)

Dynamic-scene reconstruction is almost always evaluated inside the observed time window, yet deployment settings such as AR overlays, robot interaction, and anticipatory planning need the future surface: the geometry at times beyond those captured. No standard benchmark measures this. We introduce FutureSurf, a controlled diagnostic benchmark and dataset for future-time surface reconstruction that trades scene diversity for exact future ground truth and falsification controls. A method trains on the observed first 75% of a sequence; we score its extracted per-frame surface on the held-out future by Chamfer distance, reporting absolute future CD as the primary score and the future/observed gap as a diagnostic. The dataset contains eight analytically defined controlled motions, including three falsification controls, with exact per-frame ground-truth meshes. We also provide a ground-truth-side recoverability oracle. The release includes split files, scoring code, a benchmark card, and Croissant metadata. On the controlled motions, the DG-Mesh backbone leaves a 2.7-4.1× gap even for futures predictable in principle (four of five recoverable from observed motion by a fixed rule), while the falsification controls behave as designed (the surface-invariant motion shows no gap). Beyond the contributed dataset, the gap persists across six animated DG-Mesh asset scenes and a second backbone, Deformable-3DGS (2.0-6.6×; both share a deformation-MLP temporal model). The benchmark also shows that future rendering quality and future-surface accuracy are statistically decoupled, so the novel-view-synthesis metrics the field reports do not track future geometry. The future error is structured, concentrating where the surface moves. The dataset, evaluation toolkit, and scoring code are available on Hugging Face and GitHub (https://github.com/Ricky-S/futuresurf).