# Deep Demosaicing for Polarimetric Filter Array Cameras

arXiv: https://arxiv.org/abs/2211.13732 · HTML: https://mlsanity.com/paper/2211.13732/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 45 references that are in the map.
- First posted on arXiv 24 Nov 2022.

## Builds on (most cited first, 8 of 8)

- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Deeply-Recursive Convolutional Network for Image Super-Resolution](https://mlsanity.com/paper/1511.04491/) (2015)
- [Polarimetric Multi-View Inverse Rendering](https://mlsanity.com/paper/2007.08830/) (2020)
- [Deep ChArUco: Dark ChArUco Marker Pose Estimation](https://mlsanity.com/paper/1812.03247/) (2018)
- [Deep Shape from Polarization](https://mlsanity.com/paper/1903.10210/) (2019)
- [A Sparse Representation Based Joint Demosaicing Method for Single-Chip Polarized Color Sensor](https://mlsanity.com/paper/1912.07308/) (2019)
- [An end-to-end CNN framework for polarimetric vision tasks based on polarization-parameter-constructing network](https://mlsanity.com/paper/2004.08740/) (2020)

## Built on by (most cited first, 5 of 5)

- [Efficient Unified Demosaicing for Bayer and Non-Bayer Patterned Image Sensors](https://mlsanity.com/paper/2307.10667/) (2023)
- [Learning to Deblur Polarized Images](https://mlsanity.com/paper/2402.18134/) (2024)
- [Physically Consistent Image Augmentation for Deep Learning in Mueller Matrix Polarimetry](https://mlsanity.com/paper/2411.07918/) (2024)
- [A Lensless Polarization Camera](https://mlsanity.com/paper/2603.17156/) (2026)
- [Guided Lensless Polarization Imaging](https://mlsanity.com/paper/2603.27357/) (2026)

## Abstract (arXiv)

Polarisation Filter Array (PFA) cameras allow the analysis of light polarisation state in a simple and cost-effective manner. Such filter arrays work as the Bayer pattern for colour cameras, sharing similar advantages and drawbacks. Among the others, the raw image must be demosaiced considering the local variations of the PFA and the characteristics of the imaged scene. Non-linear effects, like the cross-talk among neighbouring pixels, are difficult to explicitly model and suggest the potential advantage of a data-driven learning approach. However, the PFA cannot be removed from the sensor, making it difficult to acquire the ground-truth polarization state for training. In this work we propose a novel CNN-based model which directly demosaics the raw camera image to a per-pixel Stokes vector. Our contribution is twofold. First, we propose a network architecture composed by a sequence of Mosaiced Convolutions operating coherently with the local arrangement of the different filters. Second, we introduce a new method, employing a consumer LCD screen, to effectively acquire real-world data for training. The process is designed to be invariant by monitor gamma and external lighting conditions. We extensively compared our method against algorithmic and learning-based demosaicing techniques, obtaining a consistently lower error especially in terms of polarisation angle.