# JSTR: Joint Spatio-Temporal Reasoning for Event-based Moving Object Detection

arXiv: https://arxiv.org/abs/2403.07436 · HTML: https://mlsanity.com/paper/2403.07436/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018)
- [Event-Based Motion Segmentation by Motion Compensation](https://mlsanity.com/paper/1904.01293/) (2019)
- [Event-based Motion Segmentation with Spatio-Temporal Graph Cuts](https://mlsanity.com/paper/2012.08730/) (2020)
- [EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras](https://mlsanity.com/paper/1903.07520/) (2019)
- [SpikeMS: Deep Spiking Neural Network for Motion Segmentation](https://mlsanity.com/paper/2105.06562/) (2021)
- [Event-based Agile Object Catching with a Quadrupedal Robot](https://mlsanity.com/paper/2303.17479/) (2023)
- [Towards Bridging the Space Domain Gap for Satellite Pose Estimation using Event Sensing](https://mlsanity.com/paper/2209.11945/) (2022)
- [Event-based Object Detection and Tracking for Space Situational Awareness](https://mlsanity.com/paper/1911.08730/) (2019)
- [DOTIE - Detecting Objects through Temporal Isolation of Events using a Spiking Architecture](https://mlsanity.com/paper/2210.00975/) (2022)
- [Event-based Motion Segmentation by Cascaded Two-Level Multi-Model Fitting](https://mlsanity.com/paper/2111.03483/) (2021)

## Built on by (most cited first, 6 of 6)

- [Event-based multi-view photogrammetry for high-dynamic, high-velocity target measurement](https://mlsanity.com/paper/2506.00578/) (2025)
- [Hardware, Algorithms, and Applications of the Neuromorphic Vision Sensor: a Review](https://mlsanity.com/paper/2504.08588/) (2025)
- [Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neuroscience](https://mlsanity.com/paper/2505.05515/) (2025)
- [Iterative Event-based Motion Segmentation by Variational Contrast Maximization](https://mlsanity.com/paper/2504.18447/) (2025)
- [DarkShake-DVS: Event-based Human Action Recognition under Low-light andShaking Camera Conditions](https://mlsanity.com/paper/2605.20680/) (2026)
- [Real-time Motion Segmentation with Event-based Normal Flow](https://mlsanity.com/paper/2602.20790/) (2026)

## Abstract (arXiv)

Event-based moving object detection is a challenging task, where static background and moving object are mixed together. Typically, existing methods mainly align the background events to the same spatial coordinate system via motion compensation to distinguish the moving object. However, they neglect the potential spatial tailing effect of moving object events caused by excessive motion, which may affect the structure integrity of the extracted moving object. We discover that the moving object has a complete columnar structure in the point cloud composed of motion-compensated events along the timestamp. Motivated by this, we propose a novel joint spatio-temporal reasoning method for event-based moving object detection. Specifically, we first compensate the motion of background events using inertial measurement unit. In spatial reasoning stage, we project the compensated events into the same image coordinate, discretize the timestamp of events to obtain a time image that can reflect the motion confidence, and further segment the moving object through adaptive threshold on the time image. In temporal reasoning stage, we construct the events into a point cloud along timestamp, and use RANSAC algorithm to extract the columnar shape in the cloud for peeling off the background. Finally, we fuse the results from the two reasoning stages to extract the final moving object region. This joint spatio-temporal reasoning framework can effectively detect the moving object from motion confidence and geometric structure. Moreover, we conduct extensive experiments on various datasets to verify that the proposed method can improve the moving object detection accuracy by 13%.