# Simultaneous Acquisition of High Quality RGB Image and Polarization Information using a Sparse Polarization Sensor

arXiv: https://arxiv.org/abs/2209.13106 · HTML: https://mlsanity.com/paper/2209.13106/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 66 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 26 of 26)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [Learning a Discriminative Feature Network for Semantic Segmentation](https://mlsanity.com/paper/1804.09337/) (2018)
- [Learning Guided Convolutional Network for Depth Completion](https://mlsanity.com/paper/1908.01238/) (2019)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)
- [Polarized Reflection Removal with Perfect Alignment in the Wild](https://mlsanity.com/paper/2003.12789/) (2020)
- [Adaptive Context-Aware Multi-Modal Network for Depth Completion](https://mlsanity.com/paper/2008.10833/) (2020)
- [Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2104.06174/) (2021)
- [Deformable Kernel Networks for Joint Image Filtering](https://mlsanity.com/paper/1910.08373/) (2019)
- [Dense Depth Posterior (DDP) from Single Image and Sparse Range](https://mlsanity.com/paper/1901.10034/) (2019)
- [Deep Polarization Imaging for 3D shape and SVBRDF Acquisition](https://mlsanity.com/paper/2105.02875/) (2021)
- [Shape from Polarization for Complex Scenes in the Wild](https://mlsanity.com/paper/2112.11377/) (2021)
- [Joint Image Filtering with Deep Convolutional Networks](https://mlsanity.com/paper/1710.04200/) (2017)
- [FCFR-Net: Feature Fusion based Coarse-to-Fine Residual Learning for Depth Completion](https://mlsanity.com/paper/2012.08270/) (2020)
- [Deep attention-based classification network for robust depth prediction](https://mlsanity.com/paper/1807.03959/) (2018)
- [WESPE: Weakly Supervised Photo Enhancer for Digital Cameras](https://mlsanity.com/paper/1709.01118/) (2017)
- [Depth from a polarisation + RGB stereo pair](https://mlsanity.com/paper/1903.12061/) (2019)
- [Polarimetric Multi-View Inverse Rendering](https://mlsanity.com/paper/2007.08830/) (2020)
- [3D Human Shape Reconstruction from a Polarization Image](https://mlsanity.com/paper/2007.09268/) (2020)
- [DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network](https://mlsanity.com/paper/2108.12655/) (2021)
- [Deep Snapshot HDR Reconstruction Based on the Polarization Camera](https://mlsanity.com/paper/2105.05824/) (2021)
- [Denoising Imaging Polarimetry by an Adapted BM3D Method](https://mlsanity.com/paper/1711.04853/) (2017)
- [Polarimetric Spatio-Temporal Light Transport Probing](https://mlsanity.com/paper/2105.11609/) (2021)

## Built on by (most cited first, 9 of 9)

- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset](https://mlsanity.com/paper/2311.17396/) (2023)
- [RGB-to-Polarization Estimation: A New Task and Benchmark Study](https://mlsanity.com/paper/2505.13050/) (2025)
- [Robust Depth Enhancement via Polarization Prompt Fusion Tuning](https://mlsanity.com/paper/2404.04318/) (2024)
- [Deep Polarization Cues for Single-shot Shape and Subsurface Scattering Estimation](https://mlsanity.com/paper/2407.08149/) (2024)
- [Benchmarking Burst Super-Resolution for Polarization Images: Noise Dataset and Analysis](https://mlsanity.com/paper/2503.18705/) (2025)
- [Stokes-Informed Diffusion for Robust Linear Polarization Estimation](https://mlsanity.com/paper/2607.21239/) (2026)
- [Revisiting Shape from Polarization in the Era of Vision Foundation Models](https://mlsanity.com/paper/2603.04817/) (2026)
- [DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision](https://mlsanity.com/paper/2609.09359/) (2026)

## Abstract (arXiv)

This paper proposes a novel polarization sensor structure and network architecture to obtain a high-quality RGB image and polarization information. Conventional polarization sensors can simultaneously acquire RGB images and polarization information, but the polarizers on the sensor degrade the quality of the RGB images. There is a trade-off between the quality of the RGB image and polarization information as fewer polarization pixels reduce the degradation of the RGB image but decrease the resolution of polarization information. Therefore, we propose an approach that resolves the trade-off by sparsely arranging polarization pixels on the sensor and compensating for low-resolution polarization information with higher resolution using the RGB image as a guide. Our proposed network architecture consists of an RGB image refinement network and a polarization information compensation network. We confirmed the superiority of our proposed network in compensating the differential component of polarization intensity by comparing its performance with state-of-the-art methods for similar tasks: depth completion. Furthermore, we confirmed that our approach could simultaneously acquire higher quality RGB images and polarization information than conventional polarization sensors, resolving the trade-off between the quality of RGB images and polarization information. The baseline code and newly generated real and synthetic large-scale polarization image datasets are available for further research and development.