mlsanityData to 14 Oct 2026

Explicit Layer Modeling for Video Object Insertion and Video Layer Decomposition

arXiv 2607.25802 ↗First posted 28 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Most video editing systems still lack explicit layered video representations, limiting realistic compositing, object reuse, and consistent manipulation. This limitation is particularly evident in video object insertion and video layer decomposition, where existing methods lack direct supervision for foreground layers that capture both objects and their associated visual effects. We introduce TriLayer, a triplet video dataset containing aligned composite--background--foreground videos, where the foreground layers include both object appearance and associated visual effects. With aligned triplet supervision, TriLayer enables explicit supervised learning of layered video representations for the first time. Building on this dataset, we propose DBL-Diffusion, a dual-branch diffusion framework that jointly models scene-level RGB content and RGBA foreground layers through cross-branch interaction during denoising. We instantiate the framework in two tasks: DBL-Insert for layered object insertion, which generates explicit RGBA layers for realistic compositing and flexible post-editing, and DBL-Decompose for video layer decomposition, which recovers foreground and background layers using triplet supervision. Experiments demonstrate that explicit layer modeling substantially improves both insertion fidelity and decomposition quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

37 of 47 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  4. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
  5. 2025VACE: All-in-One Video Creation and Editingcited by 479
  6. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  7. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  8. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  9. 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 197
  10. 2020Real-Time High-Resolution Background Mattingcited by 173
  11. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 154
  12. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
25 more
  1. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 137
  2. 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
  3. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 114
  4. 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 84
  5. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  6. 2025DiffuEraser: A Diffusion Model for Video Inpaintingcited by 72
  7. 2024ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertioncited by 71
  8. 2025Se~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialistscited by 61
  9. 2021Omnimatte: Associating Objects and Their Effects in Videocited by 50
  10. 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Controlcited by 42
  11. 2021Deep Video Matting via Spatio-Temporal Alignment and Aggregationcited by 41
  12. 2024Generative Omnimatte: Learning to Decompose Video into Layerscited by 38
  13. 2025ROSE: Remove Objects with Side Effects in Videoscited by 38
  14. 2025Qwen-Image-Layered: Towards Inherent Editability via Layer Decompositioncited by 34
  15. 2025FlowDirector: Training-Free Flow Steering for Precise Text-to-Video Editingcited by 26
  16. 2021Attention-guided Temporally Coherent Video Object Mattingcited by 25
  17. 2025MatAnyone: Stable Video Matting with Consistent Memory Propagationcited by 24
  18. 2025OmnimatteZero: Fast Training-free Omnimatte with Pre-trained Video Diffusion Modelscited by 16
  19. 2023OmnimatteRF: Robust Omnimatte with 3D Background Modelingcited by 14
  20. 2024A Simple Approach to Unifying Diffusion-based Conditional Generationcited by 13
  21. 2025OmniInsert: Mask-Free Video Insertion of Any Reference via Diffusion Transformer Modelscited by 9
  22. 2025VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video Diffusion Modelscited by 8
  23. 2025LoVoRA: Text-guided and Mask-free Video Object Removal and Addition with Learnable Object-aware Localizationcited by 5
  24. 2026LaDe: Unified Multi-Layered Graphic Media Generation and Decompositioncited by 5
  25. 2025EasyOmnimatte: Taming Pretrained Inpainting Diffusion Models for End-to-End Video Layered Decompositioncited by 1

Built on by

Not cited yet.