# VSCD: Video-based Scene Change Detection in Unaligned Scenes

arXiv: https://arxiv.org/abs/2605.20821 · HTML: https://mlsanity.com/paper/2605.20821/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 20 May 2026.

## Builds on (most cited first, 18 of 18)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Fully Convolutional Siamese Networks for Change Detection](https://mlsanity.com/paper/1810.08462/) (2018)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [Weakly Supervised Silhouette-based Semantic Scene Change Detection](https://mlsanity.com/paper/1811.11985/) (2018)
- [DR-TANet: Dynamic Receptive Temporal Attention Network for Street Scene Change Detection](https://mlsanity.com/paper/2103.00879/) (2021)
- [End-to-end Active Object Tracking via Reinforcement Learning](https://mlsanity.com/paper/1705.10561/) (2017)
- [How to Reduce Change Detection to Semantic Segmentation](https://mlsanity.com/paper/2206.07557/) (2022)
- [Towards Generalizable Scene Change Detection](https://mlsanity.com/paper/2409.06214/) (2024)
- [TransVCL: Attention-enhanced Video Copy Localization Network with Flexible Supervision](https://mlsanity.com/paper/2211.13090/) (2022)
- [Zero-Shot Scene Change Detection](https://mlsanity.com/paper/2406.11210/) (2024)
- [Video Similarity and Alignment Learning on Partial Video Copy Detection](https://mlsanity.com/paper/2108.01817/) (2021)
- [Change Detection in Multi-temporal VHR Images Based on Deep Siamese Multi-scale Convolutional Networks](https://mlsanity.com/paper/1906.11479/) (2019)
- [A Similarity Alignment Model for Video Copy Segment Matching](https://mlsanity.com/paper/2305.15679/) (2023)
- [CNN Retrieval based Unsupervised Metric Learning for Near-Duplicated Video Retrieval](https://mlsanity.com/paper/2105.14566/) (2021)

## Abstract (arXiv)

Detecting what has changed in an environment is essential for long-term autonomy, yet most change detection settings assume fixed viewpoints, mild misalignment, or only a few changed objects. We introduce Video-based Scene Change Detection (VSCD), which predicts a pixel-wise change mask for each query frame, given a reference and a query RGB video of the same indoor space recorded at different times under unconstrained camera motion. The two videos are not temporally synchronized, and many object instances may appear or disappear. To study this setting, we build a large-scale benchmark with over 1.1 million frames annotated with pixel-accurate change masks, together with a real-world test set for evaluating transfer beyond simulation. We propose a query-centric multi-reference model that learns temporal matching implicitly from change-mask supervision, aligns candidate reference features to the query via local patch correspondence, and fuses per-candidate change features using frame-level and patch-level confidence before decoding a high-resolution mask once per frame. Our approach achieves state-of-the-art performance against strong image- and video-based baselines, and we validate its real-world impact by deploying it on a mobile robot for two downstream applications -- visual surveillance and object incremental learning.