# Dual Memory Aggregation Network for Event-Based Object Detection with Learnable Representation

arXiv: https://arxiv.org/abs/2303.09919 · HTML: https://mlsanity.com/paper/2303.09919/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018)
- [Event-based Asynchronous Sparse Convolutional Networks](https://mlsanity.com/paper/2003.09148/) (2020)
- [EV-SegNet: Semantic Segmentation for Event-based Cameras](https://mlsanity.com/paper/1811.12039/) (2018)
- [Asynchronous Convolutional Networks for Object Detection in Neuromorphic Cameras](https://mlsanity.com/paper/1805.07931/) (2018)
- [Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation](https://mlsanity.com/paper/2109.04871/) (2021)
- [Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object Tracking](https://mlsanity.com/paper/2002.05583/) (2020)
- [A Differentiable Recurrent Surface for Asynchronous Event-Based Data](https://mlsanity.com/paper/2001.03455/) (2020)
- [Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data](https://mlsanity.com/paper/1809.08625/) (2018)

## Built on by (most cited first, 16 of 16)

- [SODFormer: Streaming Object Detection with Transformer Using Events and Frames](https://mlsanity.com/paper/2308.04047/) (2023)
- [Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion](https://mlsanity.com/paper/2311.00436/) (2023)
- [EAS-SNN: End-to-End Adaptive Sampling and Representation for Event-based Detection with Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2403.12574/) (2024)
- [A Recurrent YOLOv8-based framework for Event-Based Object Detection](https://mlsanity.com/paper/2408.05321/) (2024)
- [Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Cameras](https://mlsanity.com/paper/2502.19630/) (2025)
- [Deep Event-based Object Detection in Autonomous Driving: A Survey](https://mlsanity.com/paper/2405.03995/) (2024)
- [Unleashing the Temporal Potential of Stereo Event Cameras for Continuous-Time 3D Object Detection](https://mlsanity.com/paper/2508.02288/) (2025)
- [Detecting Every Object from Events](https://mlsanity.com/paper/2404.05285/) (2024)
- [From Objects to Events: Unlocking Complex Visual Understanding in Object Detectors via LLM-guided Symbolic Reasoning](https://mlsanity.com/paper/2502.05843/) (2025)
- [Tracking-Assisted Object Detection with Event Cameras](https://mlsanity.com/paper/2403.18330/) (2024)
- [Dynamic Graph Induced Contour-aware Heat Conduction Network for Event-based Object Detection](https://mlsanity.com/paper/2505.12908/) (2025)
- [WD-DETR: Wavelet Denoising-Enhanced Real-Time Object Detection Transformer for Robot Perception with Event Cameras](https://mlsanity.com/paper/2506.09098/) (2025)
- [Rethinking Event-Based Object Detection through Representation-Level Temporal Aggregation and Model-Level Hypergraph Reasoning](https://mlsanity.com/paper/2605.08825/) (2026)
- [Enhancing Event-based Object Detection with Monocular Normal Maps](https://mlsanity.com/paper/2508.02127/) (2025)
- [Scalable Event-Based Video Streaming for Machines with MoQ](https://mlsanity.com/paper/2508.15003/) (2025)
- [Neuromorphic Object Detection: An In-Depth Study and Future Directions](https://mlsanity.com/paper/2607.23576/) (2026)

## Abstract (arXiv)

Event-based cameras are bio-inspired sensors that capture brightness change of every pixel in an asynchronous manner. Compared with frame-based sensors, event cameras have microsecond-level latency and high dynamic range, hence showing great potential for object detection under high-speed motion and poor illumination conditions. Due to sparsity and asynchronism nature with event streams, most of existing approaches resort to hand-crafted methods to convert event data into 2D grid representation. However, they are sub-optimal in aggregating information from event stream for object detection. In this work, we propose to learn an event representation optimized for event-based object detection. Specifically, event streams are divided into grids in the x-y-t coordinates for both positive and negative polarity, producing a set of pillars as 3D tensor representation. To fully exploit information with event streams to detect objects, a dual-memory aggregation network (DMANet) is proposed to leverage both long and short memory along event streams to aggregate effective information for object detection. Long memory is encoded in the hidden state of adaptive convLSTMs while short memory is modeled by computing spatial-temporal correlation between event pillars at neighboring time intervals. Extensive experiments on the recently released event-based automotive detection dataset demonstrate the effectiveness of the proposed method.