# Deep Metric Color Embeddings for Splicing Localization in Severely Degraded Images

arXiv: https://arxiv.org/abs/2206.10737 · HTML: https://mlsanity.com/paper/2206.10737/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 68 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [Noiseprint: a CNN-based camera model fingerprint](https://mlsanity.com/paper/1808.08396/) (2018)
- [Learning Deep Embeddings with Histogram Loss](https://mlsanity.com/paper/1611.00822/) (2016)
- [DeepFake Detection: Current Challenges and Next Steps](https://mlsanity.com/paper/2003.09234/) (2020)
- [CNN Detection of GAN-Generated Face Images based on Cross-Band Co-occurrences Analysis](https://mlsanity.com/paper/2007.12909/) (2020)
- [Fast Fourier Color Constancy](https://mlsanity.com/paper/1611.07596/) (2016)
- [Convolutional Color Constancy](https://mlsanity.com/paper/1507.00410/) (2015)
- [On the generalization of GAN image forensics](https://mlsanity.com/paper/1902.11153/) (2019)
- [Learning Spread-out Local Feature Descriptors](https://mlsanity.com/paper/1708.06320/) (2017)
- [Forensic Similarity for Digital Images](https://mlsanity.com/paper/1902.04684/) (2019)
- [Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks](https://mlsanity.com/paper/1708.00930/) (2017)
- [Localization of JPEG double compression through multi-domain convolutional neural networks](https://mlsanity.com/paper/1706.01788/) (2017)
- [On the use of Benford's law to detect GAN-generated images](https://mlsanity.com/paper/2004.07682/) (2020)
- [The Forchheim Image Database for Camera Identification in the Wild](https://mlsanity.com/paper/2011.02241/) (2020)
- [Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network](https://mlsanity.com/paper/1707.08350/) (2017)
- [Mimicking the In-Camera Color Pipeline for Camera-Aware Object Compositing](https://mlsanity.com/paper/1903.11248/) (2019)

## Abstract (arXiv)

One common task in image forensics is to detect spliced images, where multiple source images are composed to one output image. Most of the currently best performing splicing detectors leverage high-frequency artifacts. However, after an image underwent strong compression, most of the high frequency artifacts are not available anymore. In this work, we explore an alternative approach to splicing detection, which is potentially better suited for images in-the-wild, subject to strong compression and downsampling. Our proposal is to model the color formation of an image. The color formation largely depends on variations at the scale of scene objects, and is hence much less dependent on high-frequency artifacts. We learn a deep metric space that is on one hand sensitive to illumination color and camera white-point estimation, but on the other hand insensitive to variations in object color. Large distances in the embedding space indicate that two image regions either stem from different scenes or different cameras. In our evaluation, we show that the proposed embedding space outperforms the state of the art on images that have been subject to strong compression and downsampling. We confirm in two further experiments the dual nature of the metric space, namely to both characterize the acquisition camera and the scene illuminant color. As such, this work resides at the intersection of physics-based and statistical forensics with benefits from both sides.