# The Change You Want to See (Now in 3D)

arXiv: https://arxiv.org/abs/2308.10417 · HTML: https://mlsanity.com/paper/2308.10417/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 21 Aug 2023.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks](https://mlsanity.com/paper/1803.02579/) (2018)
- [Learning to Describe Differences Between Pairs of Similar Images](https://mlsanity.com/paper/1808.10584/) (2018)
- [Describing and Localizing Multiple Changes with Transformers](https://mlsanity.com/paper/2103.14146/) (2021)
- [Weakly Supervised Silhouette-based Semantic Scene Change Detection](https://mlsanity.com/paper/1811.11985/) (2018)
- [Hierarchical Paired Channel Fusion Network for Street Scene Change Detection](https://mlsanity.com/paper/2010.09925/) (2020)
- [Depth-Aware Multi-Grid Deep Homography Estimation with Contextual Correlation](https://mlsanity.com/paper/2107.02524/) (2021)

## Built on by (most cited first, 17 of 17)

- [3DGS-CD: 3D Gaussian Splatting-based Change Detection for Physical Object Rearrangement](https://mlsanity.com/paper/2411.03706/) (2024)
- [Robust Scene Change Detection Using Visual Foundation Models and Cross-Attention Mechanisms](https://mlsanity.com/paper/2409.16850/) (2024)
- [Multi-View Pose-Agnostic Change Localization with Zero Labels](https://mlsanity.com/paper/2412.03911/) (2024)
- [Step Differences in Instructional Video](https://mlsanity.com/paper/2404.16222/) (2024)
- [Progress-Aware Video Frame Captioning](https://mlsanity.com/paper/2412.02071/) (2024)
- [C-NERF: Representing Scene Changes as Directional Consistency Difference-based NeRF](https://mlsanity.com/paper/2312.02751/) (2023)
- [Gaussian Difference: Find Any Change Instance in 3D Scenes](https://mlsanity.com/paper/2502.16941/) (2025)
- [Predictive Photometric Uncertainty in Gaussian Splatting for Novel View Synthesis](https://mlsanity.com/paper/2603.22786/) (2026)
- [Has Anything Changed? 3D Change Detection by 2D Segmentation Masks](https://mlsanity.com/paper/2312.01148/) (2023)
- [ViewDelta: Scaling Scene Change Detection through Text-Conditioning](https://mlsanity.com/paper/2412.07612/) (2024)
- [SceneDiff: A Benchmark and Method for Multiview Object Change Detection](https://mlsanity.com/paper/2512.16908/) (2025)
- [Changes in Real Time: Online Scene Change Detection with Multi-View Fusion](https://mlsanity.com/paper/2511.12370/) (2025)
- [From Pixels to Primitives: Scene Change Detection in 3D Gaussian Splatting](https://mlsanity.com/paper/2605.07203/) (2026)
- [Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model](https://mlsanity.com/paper/2510.11509/) (2025)
- [PlenoCI: Plenoptic CharacterIstics for View Dependence Aware Change Classification](https://mlsanity.com/paper/2609.28930/) (2026)
- [Find the Assembly Mistakes: Error Segmentation for Industrial Applications](https://mlsanity.com/paper/2408.12945/) (2024)
- [Operational Change Detection for Geographical Information: Overview and Challenges](https://mlsanity.com/paper/2503.14109/) (2025)

## Abstract (arXiv)

The goal of this paper is to detect what has changed, if anything, between two "in the wild" images of the same 3D scene acquired from different camera positions and at different temporal instances. The open-set nature of this problem, occlusions/dis-occlusions due to the shift in viewpoint, and the lack of suitable training datasets, presents substantial challenges in devising a solution. To address this problem, we contribute a change detection model that is trained entirely on synthetic data and is class-agnostic, yet it is performant out-of-the-box on real world images without requiring fine-tuning. Our solution entails a "register and difference" approach that leverages self-supervised frozen embeddings and feature differences, which allows the model to generalise to a wide variety of scenes and domains. The model is able to operate directly on two RGB images, without requiring access to ground truth camera intrinsics, extrinsics, depth maps, point clouds, or additional before-after images. Finally, we collect and release a new evaluation dataset consisting of real-world image pairs with human-annotated differences and demonstrate the efficacy of our method. The code, datasets and pre-trained model can be found at: https://github.com/ragavsachdeva/CYWS-3D