# SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams

arXiv: https://arxiv.org/abs/2505.19487 · HTML: https://mlsanity.com/paper/2505.19487/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 98 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 30 of 45)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12514/) (2024)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Iterative Geometry Encoding Volume for Stereo Matching](https://mlsanity.com/paper/2303.06615/) (2023)
- [FoundationStereo: Zero-Shot Stereo Matching](https://mlsanity.com/paper/2501.09898/) (2025)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [Physically Grounded Vision-Language Models for Robotic Manipulation](https://mlsanity.com/paper/2309.02561/) (2023)
- [ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2312.16217/) (2023)
- [HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching](https://mlsanity.com/paper/2007.12140/) (2020)
- [Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks](https://mlsanity.com/paper/2103.12593/) (2021)
- [Event-based Stereo Visual Odometry](https://mlsanity.com/paper/2007.15548/) (2020)
- [Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching](https://mlsanity.com/paper/2403.00486/) (2024)
- [Hierarchical Discrete Distribution Decomposition for Match Density Estimation](https://mlsanity.com/paper/1812.06264/) (2018)
- [Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation](https://mlsanity.com/paper/2403.03890/) (2024)
- [Graph-Based Object Classification for Neuromorphic Vision Sensing](https://mlsanity.com/paper/1908.06648/) (2019)
- [Semi-Dense 3D Reconstruction with a Stereo Event Camera](https://mlsanity.com/paper/1807.07429/) (2018)
- [Fast and energy-efficient neuromorphic deep learning with first-spike times](https://mlsanity.com/paper/1912.11443/) (2019)
- [DEFOM-Stereo: Depth Foundation Model Based Stereo Matching](https://mlsanity.com/paper/2501.09466/) (2025)
- [Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects](https://mlsanity.com/paper/2208.03792/) (2022)
- [Omni6DPose: A Benchmark and Model for Universal 6D Object Pose Estimation and Tracking](https://mlsanity.com/paper/2406.04316/) (2024)
- [MoCha-Stereo: Motif Channel Attention Network for Stereo Matching](https://mlsanity.com/paper/2404.06842/) (2024)
- [Learning to Adapt for Stereo](https://mlsanity.com/paper/1904.02957/) (2019)
- [Neural Markov Random Field for Stereo Matching](https://mlsanity.com/paper/2403.11193/) (2024)
- [A Retina-inspired Sampling Method for Visual Texture Reconstruction](https://mlsanity.com/paper/1907.08769/) (2019)

## Built on by (most cited first, 1 of 1)

- [StereoFactory: A Unified Merging Framework for Robust Stereo Matching](https://mlsanity.com/paper/2606.17475/) (2026)

## Abstract (arXiv)

Conventional frame-based cameras often struggle with stereo depth estimation in rapidly changing scenes. In contrast, bio-inspired spike cameras emit asynchronous events at microsecond-level resolution, providing an alternative sensing modality. However, existing methods lack specialized stereo algorithms and benchmarks tailored to the spike data. To address this gap, we propose SpikeStereoNet, a brain-inspired framework and the first to estimate stereo depth directly from raw spike streams. The model fuses raw spike streams from two viewpoints and iteratively refines depth estimation through a recurrent spiking neural network (RSNN) update module. To benchmark our approach, we introduce a large-scale synthetic spike stream dataset and a real-world stereo spike dataset with dense depth annotations. SpikeStereoNet outperforms existing methods on both datasets by leveraging spike streams' ability to capture subtle edges and intensity shifts in challenging regions such as textureless surfaces and extreme lighting conditions. Furthermore, our framework exhibits strong data efficiency, maintaining high accuracy even with substantially reduced training data. The source code and datasets will be publicly available.