# Neural Spectro-polarimetric Fields

arXiv: https://arxiv.org/abs/2306.12562 · HTML: https://mlsanity.com/paper/2306.12562/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 88 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images](https://mlsanity.com/paper/2111.13679/) (2021)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Light Field Neural Rendering](https://mlsanity.com/paper/2112.09687/) (2021)
- [ReLU Fields: The Little Non-linearity That Could](https://mlsanity.com/paper/2205.10824/) (2022)
- [Differentiable Point-Based Radiance Fields for Efficient View Synthesis](https://mlsanity.com/paper/2205.14330/) (2022)
- [Deep Polarization Imaging for 3D shape and SVBRDF Acquisition](https://mlsanity.com/paper/2105.02875/) (2021)
- [Shape from Polarization for Complex Scenes in the Wild](https://mlsanity.com/paper/2112.11377/) (2021)
- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2203.12082/) (2022)
- [3D Human Shape Reconstruction from a Polarization Image](https://mlsanity.com/paper/2007.09268/) (2020)
- [Fast and Explicit Neural View Synthesis](https://mlsanity.com/paper/2107.05775/) (2021)
- [Hyperspectral Image Dataset for Benchmarking on Salient Object Detection](https://mlsanity.com/paper/1806.11314/) (2018)
- [Polarization Guided Specular Reflection Separation](https://mlsanity.com/paper/2103.11652/) (2021)
- [Sparse Ellipsometry: Portable Acquisition of Polarimetric SVBRDF and Shape with Unstructured Flash Photography](https://mlsanity.com/paper/2207.04236/) (2022)
- [Polarimetric Spatio-Temporal Light Transport Probing](https://mlsanity.com/paper/2105.11609/) (2021)

## Built on by (most cited first, 10 of 10)

- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [AONeuS: A Neural Rendering Framework for Acoustic-Optical Sensor Fusion](https://mlsanity.com/paper/2402.03309/) (2024)
- [NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics](https://mlsanity.com/paper/2411.10189/) (2024)
- [Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset](https://mlsanity.com/paper/2311.17396/) (2023)
- [MultimodalStudio: A Heterogeneous Sensor Dataset and Framework for Neural Rendering across Multiple Imaging Modalities](https://mlsanity.com/paper/2503.19673/) (2025)
- [Glossy Object Reconstruction with Cost-effective Polarized Acquisition](https://mlsanity.com/paper/2504.07025/) (2025)
- [UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields](https://mlsanity.com/paper/2506.21884/) (2025)
- [Complex-Valued Holographic Radiance Fields](https://mlsanity.com/paper/2506.08350/) (2025)
- [Learning Spectral and Polarimetric Clues for One-to-Multimodal Novel View Synthesis](https://mlsanity.com/paper/2607.02372/) (2026)
- [Spectral World Renderer: Unified 3D Hyperspectral Unmixing](https://mlsanity.com/paper/2610.19520/) (2026)

## Abstract (arXiv)

Modeling the spatial radiance distribution of light rays in a scene has been extensively explored for applications, including view synthesis. Spectrum and polarization, the wave properties of light, are often neglected due to their integration into three RGB spectral bands and their non-perceptibility to human vision. However, these properties are known to encompass substantial material and geometric information about a scene. Here, we propose to model spectro-polarimetric fields, the spatial Stokes-vector distribution of any light ray at an arbitrary wavelength. We present Neural Spectro-polarimetric Fields (NeSpoF), a neural representation that models the physically-valid Stokes vector at given continuous variables of position, direction, and wavelength. NeSpoF manages inherently noisy raw measurements, showcases memory efficiency, and preserves physically vital signals - factors that are crucial for representing the high-dimensional signal of a spectro-polarimetric field. To validate NeSpoF, we introduce the first multi-view hyperspectral-polarimetric image dataset, comprised of both synthetic and real-world scenes. These were captured using our compact hyperspectral-polarimetric imaging system, which has been calibrated for robustness against system imperfections. We demonstrate the capabilities of NeSpoF on diverse scenes.