# Beyond Static Scenes: Camera-controllable Background Generation for Human Motion

arXiv: https://arxiv.org/abs/2504.02004 · HTML: https://mlsanity.com/paper/2504.02004/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2025.

## Built on by (most cited first, 6 of 6)

- [AnimateAnywhere: Rouse the Background in Human Image Animation](https://mlsanity.com/paper/2504.19834/) (2025)
- [Explore More, Learn Better: Parallel MLLM Embeddings under Mutual Information Minimization](https://mlsanity.com/paper/2511.01588/) (2025)
- [From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Generalist Image Generation](https://mlsanity.com/paper/2608.18076/) (2026)
- [Wave-Particle (Continuous-Discrete) Dualistic Visual Tokenization for Unified Understanding and Generation](https://mlsanity.com/paper/2511.01593/) (2025)
- [Cinematic Compositing Using Character-Environment-Harmonized Video Generation Models](https://mlsanity.com/paper/2606.20233/) (2026)
- [DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics](https://mlsanity.com/paper/2607.04112/) (2026)

## Abstract (arXiv)

In this paper, we investigate the generation of new video backgrounds given a human foreground video, a camera pose, and a reference scene image. This task presents three key challenges. First, the generated background should precisely follow the camera movements corresponding to the human foreground. Second, as the camera shifts in different directions, newly revealed content should appear seamless and natural. Third, objects within the video frame should maintain consistent textures as the camera moves to ensure visual coherence. To address these challenges, we propose DynaScene, a new framework that uses camera poses extracted from the original video as an explicit control to drive background motion. Specifically, we design a multi-task learning paradigm that incorporates auxiliary tasks, namely background outpainting and scene variation, to enhance the realism of the generated backgrounds. Given the scarcity of suitable data, we constructed a large-scale, high-quality dataset tailored for this task, comprising video foregrounds, reference scene images, and corresponding camera poses. This dataset contains 200K video clips, ten times larger than existing real-world human video datasets, providing a significantly richer and more diverse training resource. Project page: https://yaomingshuai.github.io/Beyond-Static-Scenes.github.io/