# ROSE: Remove Objects with Side Effects in Videos

arXiv: https://arxiv.org/abs/2508.18633 · HTML: https://mlsanity.com/paper/2508.18633/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 47 references that are in the map.
- First posted on arXiv 26 Aug 2025.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion](https://mlsanity.com/paper/2403.06976/) (2024)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [Deep Video Inpainting](https://mlsanity.com/paper/1905.01639/) (2019)
- [FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting](https://mlsanity.com/paper/2109.02974/) (2021)
- [Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN](https://mlsanity.com/paper/1904.10247/) (2019)
- [VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control](https://mlsanity.com/paper/2503.05639/) (2025)
- [DiffuEraser: A Diffusion Model for Video Inpainting](https://mlsanity.com/paper/2501.10018/) (2025)
- [Flow-Guided Transformer for Video Inpainting](https://mlsanity.com/paper/2208.06768/) (2022)
- [Video Inpainting by Jointly Learning Temporal Structure and Spatial Details](https://mlsanity.com/paper/1806.08482/) (2018)
- [Copy-and-Paste Networks for Deep Video Inpainting](https://mlsanity.com/paper/1908.11587/) (2019)
- [CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibility](https://mlsanity.com/paper/2403.12035/) (2024)
- [OmniEraser: Remove Objects and Their Effects in Images with Paired Video-Frame Data](https://mlsanity.com/paper/2501.07397/) (2025)
- [Short-Term and Long-Term Context Aggregation Network for Video Inpainting](https://mlsanity.com/paper/2009.05721/) (2020)
- [Progressive Temporal Feature Alignment Network for Video Inpainting](https://mlsanity.com/paper/2104.03507/) (2021)
- [Towards Language-Driven Video Inpainting via Multimodal Large Language Models](https://mlsanity.com/paper/2401.10226/) (2024)
- [SmartEraser: Remove Anything from Images using Masked-Region Guidance](https://mlsanity.com/paper/2501.08279/) (2025)

## Built on by (most cited first, 30 of 38)

- [EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing](https://mlsanity.com/paper/2603.19224/) (2026)
- [PISCO: Precise Video Instance Insertion with Sparse Control](https://mlsanity.com/paper/2602.08277/) (2026)
- [VOID: Video Object and Interaction Deletion](https://mlsanity.com/paper/2604.02296/) (2026)
- [Aurora: Unified Video Editing with a Tool-Using Agent](https://mlsanity.com/paper/2605.18748/) (2026)
- [Object-WIPER : Training-Free Object and Associated Effect Removal in Videos](https://mlsanity.com/paper/2601.06391/) (2026)
- [LoVoRA: Text-guided and Mask-free Video Object Removal and Addition with Learnable Object-aware Localization](https://mlsanity.com/paper/2512.02933/) (2025)
- [From Ideal to Real: Stable Video Object Removal under Imperfect Conditions](https://mlsanity.com/paper/2603.09283/) (2026)
- [AVControl: Efficient Framework for Training Audio-Visual Controls](https://mlsanity.com/paper/2603.24793/) (2026)
- [NOVA: Sparse Control, Dense Synthesis for Pair-Free Video Editing](https://mlsanity.com/paper/2603.02802/) (2026)
- [VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects](https://mlsanity.com/paper/2604.16272/) (2026)
- [Over++: Generative Video Compositing for Layer Interaction Effects](https://mlsanity.com/paper/2512.19661/) (2025)
- [From Understanding to Erasing: Towards Complete and Stable Video Object Removal](https://mlsanity.com/paper/2604.01693/) (2026)
- [Physics-Aware Video Instance Removal Benchmark](https://mlsanity.com/paper/2604.05898/) (2026)
- [UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models](https://mlsanity.com/paper/2608.04701/) (2026)
- [YOSE: You Only Select Essential Tokens for Efficient DiT-based Video Object Removal](https://mlsanity.com/paper/2604.27322/) (2026)
- [PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media](https://mlsanity.com/paper/2605.14534/) (2026)
- [Learning Stochastic Bridges for Video Object Removal via Video-to-Video Translation](https://mlsanity.com/paper/2601.12066/) (2026)
- [LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing](https://mlsanity.com/paper/2604.17021/) (2026)
- [MiVE: Multiscale Vision-language features for reference-guided video Editing](https://mlsanity.com/paper/2605.14664/) (2026)
- [DualEraser: Joint Video Object and Effect Removal via Balanced Text-Mask Guidance and Decoupled Locator-Preserver](https://mlsanity.com/paper/2605.30045/) (2026)
- [From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting](https://mlsanity.com/paper/2607.14976/) (2026)
- [OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing](https://mlsanity.com/paper/2608.05049/) (2026)
- [BeyondMasks: Evaluating Causal and Physical Consistency in Video Object Removal](https://mlsanity.com/paper/2608.20107/) (2026)
- [A Study of the Design Space of Motion and Disocclusion Control in Video Generation](https://mlsanity.com/paper/2512.13392/) (2025)
- [InsertAnywhere: Geometrically Grounded and Optics-Aware Video Object Insertion](https://mlsanity.com/paper/2512.17504/) (2025)
- [Search2Motion: Training-Free Object-Level Motion Control via Attention-Consensus Search](https://mlsanity.com/paper/2603.16711/) (2026)
- [SceneForge: Structured World Supervision from 3D Interventions](https://mlsanity.com/paper/2605.14399/) (2026)
- [AlbedoEdit: Unified Instance-Level Video Editing with Albedo Guidance](https://mlsanity.com/paper/2606.01362/) (2026)
- [Explicit Layer Modeling for Video Object Insertion and Video Layer Decomposition](https://mlsanity.com/paper/2607.25802/) (2026)
- [EffectLearner: World-Aware Object-Effect Reasoning for Real-World Video Object Removal](https://mlsanity.com/paper/2608.05565/) (2026)

## Abstract (arXiv)

Video object removal has achieved advanced performance due to the recent success of video generative models. However, when addressing the side effects of objects, e.g., their shadows and reflections, existing works struggle to eliminate these effects for the scarcity of paired video data as supervision. This paper presents ROSE, termed Remove Objects with Side Effects, a framework that systematically studies the object's effects on environment, which can be categorized into five common cases: shadows, reflections, light, translucency and mirror. Given the challenges of curating paired videos exhibiting the aforementioned effects, we leverage a 3D rendering engine for synthetic data generation. We carefully construct a fully-automatic pipeline for data preparation, which simulates a large-scale paired dataset with diverse scenes, objects, shooting angles, and camera trajectories. ROSE is implemented as an video inpainting model built on diffusion transformer. To localize all object-correlated areas, the entire video is fed into the model for reference-based erasing. Moreover, additional supervision is introduced to explicitly predict the areas affected by side effects, which can be revealed through the differential mask between the paired videos. To fully investigate the model performance on various side effect removal, we presents a new benchmark, dubbed ROSE-Bench, incorporating both common scenarios and the five special side effects for comprehensive evaluation. Experimental results demonstrate that ROSE achieves superior performance compared to existing video object erasing models and generalizes well to real-world video scenarios. The project page is https://rose2025-inpaint.github.io/.