# SoDaCam: Software-defined Cameras via Single-Photon Imaging

arXiv: https://arxiv.org/abs/2309.00066 · HTML: https://mlsanity.com/paper/2309.00066/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 100 references that are in the map.
- First posted on arXiv 31 Aug 2023.

## Builds on (most cited first, 29 of 29)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [Rank Minimization for Snapshot Compressive Imaging](https://mlsanity.com/paper/1807.07837/) (2018)
- [Event-aided Direct Sparse Odometry](https://mlsanity.com/paper/2204.07640/) (2022)
- [Object Tracking by Jointly Exploiting Frame and Event Domain](https://mlsanity.com/paper/2109.09052/) (2021)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [Learning Dense and Continuous Optical Flow from an Event Camera](https://mlsanity.com/paper/2211.09078/) (2022)
- [Coded aperture compressive temporal imaging](https://mlsanity.com/paper/1302.2575/) (2013)
- [Plug-and-Play Algorithms for Video Snapshot Compressive Imaging](https://mlsanity.com/paper/2101.04822/) (2021)
- [Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot Compressive Imaging](https://mlsanity.com/paper/2109.06548/) (2021)
- [MetaSCI: Scalable and Adaptive Reconstruction for Video Compressive Sensing](https://mlsanity.com/paper/2103.01786/) (2021)
- [A megapixel time-gated SPAD image sensor for 2D and 3D imaging applications](https://mlsanity.com/paper/1912.12910/) (2019)
- [Quanta Burst Photography](https://mlsanity.com/paper/2006.11840/) (2020)
- [High Flux Passive Imaging with Single-Photon Sensors](https://mlsanity.com/paper/1902.10190/) (2019)
- [Passive Inter-Photon Imaging](https://mlsanity.com/paper/2104.00059/) (2021)
- [Unraveling the paradox of intensity-dependent DVS pixel noise](https://mlsanity.com/paper/2109.08640/) (2021)
- [HDR Video Reconstruction with Tri-Exposure Quad-Bayer Sensors](https://mlsanity.com/paper/2103.10982/) (2021)
- [Motion Adaptive Deblurring with Single-Photon Cameras](https://mlsanity.com/paper/2012.07931/) (2020)
- [Ensemble learning priors unfolding for scalable Snapshot Compressive Sensing](https://mlsanity.com/paper/2201.10419/) (2022)
- [A Unified Framework for Compressive Video Recovery from Coded Exposure Techniques](https://mlsanity.com/paper/2011.05532/) (2020)

## Built on by (most cited first, 10 of 10)

- [Generalized Event Cameras](https://mlsanity.com/paper/2407.02683/) (2024)
- [Time-Resolved MNIST Dataset for Single-Photon Recognition](https://mlsanity.com/paper/2410.16744/) (2024)
- [Streaming quanta sensors for online, high-performance imaging and vision](https://mlsanity.com/paper/2406.00859/) (2024)
- [Radiance Fields from Photons](https://mlsanity.com/paper/2407.09386/) (2024)
- [Photon Inhibition for Energy-Efficient Single-Photon Imaging](https://mlsanity.com/paper/2409.18337/) (2024)
- [Spiking Neural Network Enhanced Hand Gesture Recognition Using Low-Cost Single-photon Avalanche Diode Array](https://mlsanity.com/paper/2402.05441/) (2024)
- [Hardware-aware Coding Function Design for Compressive Single-Photon 3D Cameras](https://mlsanity.com/paper/2510.12123/) (2025)
- [Accurate Simulation Pipeline for Passive Single-Photon Imaging](https://mlsanity.com/paper/2601.12850/) (2026)
- [Computer Vision with a Superpixelation Camera](https://mlsanity.com/paper/2603.26900/) (2026)
- [Optical Flow from Photons](https://mlsanity.com/paper/2608.00499/) (2026)

## Abstract (arXiv)

Reinterpretable cameras are defined by their post-processing capabilities that exceed traditional imaging. We present "SoDaCam" that provides reinterpretable cameras at the granularity of photons, from photon-cubes acquired by single-photon devices. Photon-cubes represent the spatio-temporal detections of photons as a sequence of binary frames, at frame-rates as high as 100 kHz. We show that simple transformations of the photon-cube, or photon-cube projections, provide the functionality of numerous imaging systems including: exposure bracketing, flutter shutter cameras, video compressive systems, event cameras, and even cameras that move during exposure. Our photon-cube projections offer the flexibility of being software-defined constructs that are only limited by what is computable, and shot-noise. We exploit this flexibility to provide new capabilities for the emulated cameras. As an added benefit, our projections provide camera-dependent compression of photon-cubes, which we demonstrate using an implementation of our projections on a novel compute architecture that is designed for single-photon imaging.