# Occlusion-Aware Physics-Semantic Keyframe Selection for Robust Video Editing

arXiv: https://arxiv.org/abs/2605.23192 · HTML: https://mlsanity.com/paper/2605.23192/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 22 May 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 27 of 27)

- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Tasks](https://mlsanity.com/paper/2403.14468/) (2024)
- [VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control](https://mlsanity.com/paper/2503.05639/) (2025)
- [Consistent Video-to-Video Transfer Using Synthetic Dataset](https://mlsanity.com/paper/2311.00213/) (2023)
- [Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset](https://mlsanity.com/paper/2510.15742/) (2025)
- [InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Construction](https://mlsanity.com/paper/2503.20287/) (2025)
- [EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning](https://mlsanity.com/paper/2509.20360/) (2025)
- [OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editing](https://mlsanity.com/paper/2512.07826/) (2025)
- [VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Control](https://mlsanity.com/paper/2501.01427/) (2025)
- [VideoDirector: Precise Video Editing via Text-to-Video Models](https://mlsanity.com/paper/2411.17592/) (2024)
- [In-Context Learning with Unpaired Clips for Instruction-based Video Editing](https://mlsanity.com/paper/2510.14648/) (2025)
- [Occlusion-Aware Video Object Inpainting](https://mlsanity.com/paper/2108.06765/) (2021)
- [Region-Constraint In-Context Generation for Instructional Video Editing](https://mlsanity.com/paper/2512.17650/) (2025)
- [Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing](https://mlsanity.com/paper/2602.08820/) (2026)
- [MTV-Inpaint: Multi-Task Long Video Inpainting](https://mlsanity.com/paper/2503.11412/) (2025)
- [UniPaint: Unified Space-time Video Inpainting via Mixture-of-Experts](https://mlsanity.com/paper/2412.06340/) (2024)
- [Get In Video: Add Anything You Want to the Video](https://mlsanity.com/paper/2503.06268/) (2025)
- [O-DisCo-Edit: Object Distortion Control for Unified Realistic Video Editing](https://mlsanity.com/paper/2509.01596/) (2025)
- [PISCO: Precise Video Instance Insertion with Sparse Control](https://mlsanity.com/paper/2602.08277/) (2026)
- [SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing](https://mlsanity.com/paper/2603.19228/) (2026)
- [Point2Insert: Video Object Insertion via Sparse Point Guidance](https://mlsanity.com/paper/2602.04167/) (2026)
- [TRACE: Object Motion Editing in Videos with First-Frame Trajectory Guidance](https://mlsanity.com/paper/2603.25707/) (2026)

## Abstract (arXiv)

Video editing has recently achieved remarkable progress with diffusion-based generative models, enabling diverse object-level manipulations from natural language instructions. However, existing methods often struggle under occlusion, viewpoint changes, and fast object motion, where unreliable visual observations lead to inaccurate localization, temporal flickering, and inconsistent edits. In this work, we identify the absence of reliable visual anchors as a fundamental bottleneck in occlusion-robust video editing. To address this issue, we propose an occlusion-aware physics-semantic keyframe selection framework that automatically identifies an optimal anchor frame for downstream editing. Specifically, our method evaluates candidate frames from three complementary perspectives: structural completeness for avoiding truncated observations, cycle-consistent tracking stability for measuring physical reliability, and vision-language-based attribute visibility for ensuring semantic clarity. The selected keyframe is then propagated through bidirectional tracking to generate dense spatiotemporal masks, which are used as auxiliary supervision for a diffusion-based video editing backbone. By transforming occlusion handling from explicit reconstruction into reliable anchor selection, our framework enables precise and temporally consistent editing without requiring manual annotations. Extensive experiments on challenging video editing benchmarks demonstrate the effectiveness and high-quality performance of our method.