# Efficient Unified Demosaicing for Bayer and Non-Bayer Patterned Image Sensors

arXiv: https://arxiv.org/abs/2307.10667 · HTML: https://mlsanity.com/paper/2307.10667/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 83 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Builds on (most cited first, 30 of 42)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Multi-Scale Boosted Dehazing Network with Dense Feature Fusion](https://mlsanity.com/paper/2004.13388/) (2020)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [Meta-SR: A Magnification-Arbitrary Network for Super-Resolution](https://mlsanity.com/paper/1903.00875/) (2019)
- [Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images](https://mlsanity.com/paper/2101.02824/) (2021)
- [Deep Generalized Unfolding Networks for Image Restoration](https://mlsanity.com/paper/2204.13348/) (2022)
- [Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification](https://mlsanity.com/paper/2012.00417/) (2020)
- [Replacing Mobile Camera ISP with a Single Deep Learning Model](https://mlsanity.com/paper/2002.05509/) (2020)
- [DeepISP: Towards Learning an End-to-End Image Processing Pipeline](https://mlsanity.com/paper/1801.06724/) (2018)
- [Spatially-Adaptive Image Restoration using Distortion-Guided Networks](https://mlsanity.com/paper/2108.08617/) (2021)
- [Offline Meta-Reinforcement Learning with Online Self-Supervision](https://mlsanity.com/paper/2107.03974/) (2021)
- [Realistic Blur Synthesis for Learning Image Deblurring](https://mlsanity.com/paper/2202.08771/) (2022)
- [BokehMe: When Neural Rendering Meets Classical Rendering](https://mlsanity.com/paper/2206.12614/) (2022)
- [Dirty Pixels: Towards End-to-End Image Processing and Perception](https://mlsanity.com/paper/1701.06487/) (2017)
- [Gradients of Counterfactuals](https://mlsanity.com/paper/1611.02639/) (2016)
- [Deep Image Demosaicking using a Cascade of Convolutional Residual Denoising Networks](https://mlsanity.com/paper/1803.05215/) (2018)
- [Joint Noise-Tolerant Learning and Meta Camera Shift Adaptation for Unsupervised Person Re-Identification](https://mlsanity.com/paper/2103.04618/) (2021)

## Built on by (most cited first, 6 of 6)

- [MambaSCI: Efficient Mamba-UNet for Quad-Bayer Patterned Video Snapshot Compressive Imaging](https://mlsanity.com/paper/2410.14214/) (2024)
- [An Extensible and Lightweight Unified Architecture for Demosaicing Pixel-bin Image Sensors](https://mlsanity.com/paper/2606.13136/) (2026)
- [Multispectral Demosaicing via Dual Cameras](https://mlsanity.com/paper/2503.22026/) (2025)
- [Frequency Enhancement for Image Demosaicking](https://mlsanity.com/paper/2503.15800/) (2025)
- [Efficient Deep Demosaicing with Spatially Downsampled Isotropic Networks](https://mlsanity.com/paper/2601.00703/) (2026)
- [SegDem: Segmentation helps Demosaicing](https://mlsanity.com/paper/2608.07916/) (2026)

## Abstract (arXiv)

As the physical size of recent CMOS image sensors (CIS) gets smaller, the latest mobile cameras are adopting unique non-Bayer color filter array (CFA) patterns (e.g., Quad, Nona, QxQ), which consist of homogeneous color units with adjacent pixels. These non-Bayer sensors are superior to conventional Bayer CFA thanks to their changeable pixel-bin sizes for different light conditions but may introduce visual artifacts during demosaicing due to their inherent pixel pattern structures and sensor hardware characteristics. Previous demosaicing methods have primarily focused on Bayer CFA, necessitating distinct reconstruction methods for non-Bayer patterned CIS with various CFA modes under different lighting conditions. In this work, we propose an efficient unified demosaicing method that can be applied to both conventional Bayer RAW and various non-Bayer CFAs' RAW data in different operation modes. Our Knowledge Learning-based demosaicing model for Adaptive Patterns, namely KLAP, utilizes CFA-adaptive filters for only 1% key filters in the network for each CFA, but still manages to effectively demosaic all the CFAs, yielding comparable performance to the large-scale models. Furthermore, by employing meta-learning during inference (KLAP-M), our model is able to eliminate unknown sensor-generic artifacts in real RAW data, effectively bridging the gap between synthetic images and real sensor RAW. Our KLAP and KLAP-M methods achieved state-of-the-art demosaicing performance in both synthetic and real RAW data of Bayer and non-Bayer CFAs.