# Raw Bayer Pattern Image Synthesis for Computer Vision-oriented Image Signal Processing Pipeline Design

arXiv: https://arxiv.org/abs/2110.12823 · HTML: https://mlsanity.com/paper/2110.12823/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks](https://mlsanity.com/paper/1703.05192/) (2017)
- [Reconfiguring the Imaging Pipeline for Computer Vision](https://mlsanity.com/paper/1705.04352/) (2017)
- [Neural Network Generalization: The impact of camera parameters](https://mlsanity.com/paper/1912.03604/) (2019)
- [Gradient-based Feature Extraction From Raw Bayer Pattern Images](https://mlsanity.com/paper/2004.02429/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Learnable Color Correction Matrix for RAW Reconstruction](https://mlsanity.com/paper/2409.02497/) (2024)
- [Raw Instinct: Trust Your Classifiers and Skip the Conversion](https://mlsanity.com/paper/2403.14439/) (2024)
- [Leveraging Motion Estimation for Efficient Bayer-Domain Video Convolutional Networks](https://mlsanity.com/paper/2501.15119/) (2025)
- [R3: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain](https://mlsanity.com/paper/2509.24022/) (2025)

## Abstract (arXiv)

In this paper, we propose a method to add constraints that are un-formulatable in generative adversarial networks (GAN)-based arbitrary size RAW Bayer image generation. It is shown theoretically that by using the transformed data in GAN training, it is able to improve the learning of the original data distribution, owing to the invariant of Jensen-Shannon (JS) divergence between two distributions under invertible and differentiable transformation. Benefiting from the proposed method, RAW Bayer pattern images can be generated by configuring the transformation as demosaicing. It is shown that by adding another transformation, the proposed method is able to synthesize high-quality RAW Bayer images with arbitrary size. Experimental results show that images generated by the proposed method outperform the existing methods in the Fr'echet inception distance (FID) score, peak signal to noise ratio (PSNR), and mean structural similarity (MSSIM), and the training process is more stable. To the best knowledge of the authors, there is no open-source, large-scale image dataset in the RAW Bayer domain, which is crucial for research works aiming to explore the image signal processing (ISP) pipeline design for computer vision tasks. Converting the existing commonly used color image datasets to their corresponding RAW Bayer versions, the proposed method can be a promising solution to the RAW image dataset problem. We also show in the experiments that, by training object detection frameworks using the synthesized RAW Bayer images, they can be used in an end-to-end manner (from RAW images to vision tasks) with negligible performance degradation.