# Event-based Synthetic Aperture Imaging with a Hybrid Network

arXiv: https://arxiv.org/abs/2103.02376 · HTML: https://mlsanity.com/paper/2103.02376/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios](https://mlsanity.com/paper/1709.06310/) (2017)
- [Event Enhanced High-Quality Image Recovery](https://mlsanity.com/paper/2007.08336/) (2020)
- [DeOccNet: Learning to See Through Foreground Occlusions in Light Fields](https://mlsanity.com/paper/1912.04459/) (2019)
- [Towards Scalable, Efficient and Accurate Deep Spiking Neural Networks with Backward Residual Connections, Stochastic Softmax and Hybridization](https://mlsanity.com/paper/1910.13931/) (2019)

## Built on by (most cited first, 17 of 17)

- [AEGNN: Asynchronous Event-based Graph Neural Networks](https://mlsanity.com/paper/2203.17149/) (2022)
- [Event-based Video Reconstruction via Potential-assisted Spiking Neural Network](https://mlsanity.com/paper/2201.10943/) (2022)
- [Deep Learning for HDR Imaging: State-of-the-Art and Future Trends](https://mlsanity.com/paper/2110.10394/) (2021)
- [Generalizing Event-Based Motion Deblurring in Real-World Scenarios](https://mlsanity.com/paper/2308.05932/) (2023)
- [Are High-Resolution Event Cameras Really Needed?](https://mlsanity.com/paper/2203.14672/) (2022)
- [LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising](https://mlsanity.com/paper/2405.19718/) (2024)
- [E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning](https://mlsanity.com/paper/2311.18433/) (2023)
- [Learning to See Through with Events](https://mlsanity.com/paper/2212.02219/) (2022)
- [A Dynamic Graph CNN with Cross-Representation Distillation for Event-Based Recognition](https://mlsanity.com/paper/2302.04177/) (2023)
- [Efficient Event-Based Semantic Segmentation via Exploiting Frame-Event Fusion: A Hybrid Neural Network Approach](https://mlsanity.com/paper/2507.03765/) (2025)
- [Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views](https://mlsanity.com/paper/2307.00821/) (2023)
- [DeblurSR: Event-Based Motion Deblurring Under the Spiking Representation](https://mlsanity.com/paper/2303.08977/) (2023)
- [Seeing Behind Dynamic Occlusions with Event Cameras](https://mlsanity.com/paper/2307.15829/) (2023)
- [HDR Imaging for Dynamic Scenes with Events](https://mlsanity.com/paper/2404.03210/) (2024)
- [Event Camera Guided Visual Media Restoration & 3D Reconstruction: A Survey](https://mlsanity.com/paper/2509.09971/) (2025)
- [Dual Exposure Stereo for Extended Dynamic Range 3D Imaging](https://mlsanity.com/paper/2412.02351/) (2024)
- [Event Cameras versus Frame Cameras for Synthetic-Aperture Imaging: A Simulation Study](https://mlsanity.com/paper/2610.20309/) (2026)

## Abstract (arXiv)

Synthetic aperture imaging (SAI) is able to achieve the see through effect by blurring out the off-focus foreground occlusions and reconstructing the in-focus occluded targets from multi-view images. However, very dense occlusions and extreme lighting conditions may bring significant disturbances to the SAI based on conventional frame-based cameras, leading to performance degeneration. To address these problems, we propose a novel SAI system based on the event camera which can produce asynchronous events with extremely low latency and high dynamic range. Thus, it can eliminate the interference of dense occlusions by measuring with almost continuous views, and simultaneously tackle the over/under exposure problems. To reconstruct the occluded targets, we propose a hybrid encoder-decoder network composed of spiking neural networks (SNNs) and convolutional neural networks (CNNs). In the hybrid network, the spatio-temporal information of the collected events is first encoded by SNN layers, and then transformed to the visual image of the occluded targets by a style-transfer CNN decoder. Through experiments, the proposed method shows remarkable performance in dealing with very dense occlusions and extreme lighting conditions, and high quality visual images can be reconstructed using pure event data.