# SIDAR: Synthetic Image Dataset for Alignment & Restoration

arXiv: https://arxiv.org/abs/2305.12036 · HTML: https://mlsanity.com/paper/2305.12036/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 24 of 24)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise](https://mlsanity.com/paper/2208.09392/) (2022)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences](https://mlsanity.com/paper/1912.05524/) (2019)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [PDC-Net+: Enhanced Probabilistic Dense Correspondence Network](https://mlsanity.com/paper/2109.13912/) (2021)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [RANSAC-Flow: generic two-stage image alignment](https://mlsanity.com/paper/2004.01526/) (2020)
- [Iterative Deep Homography Estimation](https://mlsanity.com/paper/2203.15982/) (2022)
- [CONSAC: Robust Multi-Model Fitting by Conditional Sample Consensus](https://mlsanity.com/paper/2001.02643/) (2020)
- [A Large Scale Homography Benchmark](https://mlsanity.com/paper/2302.09997/) (2023)
- [An Empirical Review of Deep Learning Frameworks for Change Detection: Model Design, Experimental Frameworks, Challenges and Research Needs](https://mlsanity.com/paper/2105.01342/) (2021)

## Built on by (most cited first, 2 of 2)

- [Computer User Interface Understanding. A New Dataset and a Learning Framework](https://mlsanity.com/paper/2403.10170/) (2024)
- [ConDL: Detector-Free Dense Image Matching](https://mlsanity.com/paper/2408.02766/) (2024)

## Abstract (arXiv)

Image alignment and image restoration are classical computer vision tasks. However, there is still a lack of datasets that provide enough data to train and evaluate end-to-end deep learning models. Obtaining ground-truth data for image alignment requires sophisticated structure-from-motion methods or optical flow systems that often do not provide enough data variance, i.e., typically providing a high number of image correspondences, while only introducing few changes of scenery within the underlying image sequences. Alternative approaches utilize random perspective distortions on existing image data. However, this only provides trivial distortions, lacking the complexity and variance of real-world scenarios. Instead, our proposed data augmentation helps to overcome the issue of data scarcity by using 3D rendering: images are added as textures onto a plane, then varying lighting conditions, shadows, and occlusions are added to the scene. The scene is rendered from multiple viewpoints, generating perspective distortions more consistent with real-world scenarios, with homographies closely resembling those of camera projections rather than randomized homographies. For each scene, we provide a sequence of distorted images with corresponding occlusion masks, homographies, and ground-truth labels. The resulting dataset can serve as a training and evaluation set for a multitude of tasks involving image alignment and artifact removal, such as deep homography estimation, dense image matching, 2D bundle adjustment, inpainting, shadow removal, denoising, content retrieval, and background subtraction. Our data generation pipeline is customizable and can be applied to any existing dataset, serving as a data augmentation to further improve the feature learning of any existing method.