# Predicting 3D representations for Dynamic Scenes

arXiv: https://arxiv.org/abs/2501.16617 · HTML: https://mlsanity.com/paper/2501.16617/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2025.

## Built on by (most cited first, 2 of 2)

- [4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos](https://mlsanity.com/paper/2506.08015/) (2025)
- [FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Images](https://mlsanity.com/paper/2606.30347/) (2026)

## Abstract (arXiv)

We present a novel framework for dynamic radiance field prediction given monocular video streams. Unlike previous methods that primarily focus on predicting future frames, our method goes a step further by generating explicit 3D representations of the dynamic scene. The framework builds on two core designs. First, we adopt an ego-centric unbounded triplane to explicitly represent the dynamic physical world. Second, we develop a 4D-aware transformer to aggregate features from monocular videos to update the triplane. Coupling these two designs enables us to train the proposed model with large-scale monocular videos in a self-supervised manner. Our model achieves top results in dynamic radiance field prediction on NVIDIA dynamic scenes, demonstrating its strong performance on 4D physical world modeling. Besides, our model shows a superior generalizability to unseen scenarios. Notably, we find that our approach emerges capabilities for geometry and semantic learning.