# Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset

arXiv: https://arxiv.org/abs/2311.17396 · HTML: https://mlsanity.com/paper/2311.17396/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 73 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Polarized Reflection Removal with Perfect Alignment in the Wild](https://mlsanity.com/paper/2003.12789/) (2020)
- [Deep Polarization Imaging for 3D shape and SVBRDF Acquisition](https://mlsanity.com/paper/2105.02875/) (2021)
- [Shape from Polarization for Complex Scenes in the Wild](https://mlsanity.com/paper/2112.11377/) (2021)
- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [KBNet: Kernel Basis Network for Image Restoration](https://mlsanity.com/paper/2303.02881/) (2023)
- [3D Human Shape Reconstruction from a Polarization Image](https://mlsanity.com/paper/2007.09268/) (2020)
- [Hyperspectral Image Dataset for Benchmarking on Salient Object Detection](https://mlsanity.com/paper/1806.11314/) (2018)
- [Polarimetric Pose Prediction](https://mlsanity.com/paper/2112.03810/) (2021)
- [Polarization Guided Specular Reflection Separation](https://mlsanity.com/paper/2103.11652/) (2021)
- [Neural Spectro-polarimetric Fields](https://mlsanity.com/paper/2306.12562/) (2023)
- [Simultaneous Acquisition of High Quality RGB Image and Polarization Information using a Sparse Polarization Sensor](https://mlsanity.com/paper/2209.13106/) (2022)
- [Deep Shape from Polarization](https://mlsanity.com/paper/1903.10210/) (2019)

## Built on by (most cited first, 8 of 8)

- [PolarFree: Polarization-based Reflection-free Imaging](https://mlsanity.com/paper/2503.18055/) (2025)
- [RGB-to-Polarization Estimation: A New Task and Benchmark Study](https://mlsanity.com/paper/2505.13050/) (2025)
- [Event Ellipsometer: Event-based Mueller-Matrix Video Imaging](https://mlsanity.com/paper/2411.17313/) (2024)
- [Benchmarking Burst Super-Resolution for Polarization Images: Noise Dataset and Analysis](https://mlsanity.com/paper/2503.18705/) (2025)
- [NeST: Neural Stress Tensor Tomography by leveraging 3D Photoelasticity](https://mlsanity.com/paper/2406.10212/) (2024)
- [Stokes-Informed Diffusion for Robust Linear Polarization Estimation](https://mlsanity.com/paper/2607.21239/) (2026)
- [PolarVSR: A Unified Framework and Benchmark for Continuous Space-Time Polarization Video Reconstruction](https://mlsanity.com/paper/2605.10275/) (2026)
- [DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision](https://mlsanity.com/paper/2609.09359/) (2026)

## Abstract (arXiv)

Image datasets are essential not only in validating existing methods in computer vision but also in developing new methods. Most existing image datasets focus on trichromatic intensity images to mimic human vision. However, polarization and spectrum, the wave properties of light that animals in harsh environments and with limited brain capacity often rely on, remain underrepresented in existing datasets. Although spectro-polarimetric datasets exist, these datasets have insufficient object diversity, limited illumination conditions, linear-only polarization data, and inadequate image count. Here, we introduce two spectro-polarimetric datasets: trichromatic Stokes images and hyperspectral Stokes images. These novel datasets encompass both linear and circular polarization; they introduce multiple spectral channels; and they feature a broad selection of real-world scenes. With our dataset in hand, we analyze the spectro-polarimetric image statistics, develop efficient representations of such high-dimensional data, and evaluate spectral dependency of shape-from-polarization methods. As such, the proposed dataset promises a foundation for data-driven spectro-polarimetric imaging and vision research. Dataset and code will be publicly available.