# Optical Flow from Photons

arXiv: https://arxiv.org/abs/2608.00499 · HTML: https://mlsanity.com/paper/2608.00499/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 1 Aug 2026.

## Builds on (most cited first, 23 of 23)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [Learning Motion Patterns in Videos](https://mlsanity.com/paper/1612.07217/) (2016)
- [Dense Continuous-Time Optical Flow from Events and Frames](https://mlsanity.com/paper/2203.13674/) (2022)
- [WAFT: Warping-Alone Field Transforms for Optical Flow](https://mlsanity.com/paper/2506.21526/) (2025)
- [Optical Flow Estimation for Spiking Camera](https://mlsanity.com/paper/2110.03916/) (2021)
- [Quanta Burst Photography](https://mlsanity.com/paper/2006.11840/) (2020)
- [End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving](https://mlsanity.com/paper/2104.08862/) (2021)
- [Optical Flow based Visual Potential Field for Autonomous Driving](https://mlsanity.com/paper/2006.00168/) (2020)
- [Efficient Optical flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone](https://mlsanity.com/paper/1612.06702/) (2016)
- [Motion Adaptive Deblurring with Single-Photon Cameras](https://mlsanity.com/paper/2012.07931/) (2020)
- [SoDaCam: Software-defined Cameras via Single-Photon Imaging](https://mlsanity.com/paper/2309.00066/) (2023)
- [Coherent Motion Segmentation in Moving Camera Videos using Optical Flow Orientations](https://mlsanity.com/paper/1511.01619/) (2015)

## Abstract (arXiv)

Optical flow remains challenging in high-speed and low-light scenes, where the limited frame rate and sensitivity of conventional cameras lead to motion blur and underexposure. Single-photon avalanche diode (SPAD) cameras offer single-photon sensitivity and extremely fine temporal sampling. However, individual slices in these high FPS binary photon streams are too sparse for dense correspondence. Temporal aggregation can provide the spatial cues required by optical flow, but accumulating photons at fixed coordinates blurs moving structures. Motion-aware aggregation can reduce this blur, yet it depends on the flow being estimated. To address this dependency, we propose QuantaFlow, the first method for dense optical flow directly from SPAD streams. Instead of constructing a fixed input representation, QuantaFlow embeds SPAD representation construction into iterative flow refinement. At each iteration, the current flow coarsely aligns the slices within the source and target sub-streams. A photon-flux transformation then constructs multi-scale representations containing intensity and structural cues, while adaptive multi-scale fusion balances photon noise and residual motion blur at each pixel. The fused representations drive a feature-warping flow update, and the refined flow guides representation construction in the next iteration. We further construct a synthetic dataset for SPAD optical-flow training and evaluation. Experiments on the synthetic dataset and real-world SPAD data demonstrate the effectiveness and generalization of QuantaFlow.