# Instantaneous Perception of Moving Objects in 3D

arXiv: https://arxiv.org/abs/2405.02781 · HTML: https://mlsanity.com/paper/2405.02781/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 62 references that are in the map.
- First posted on arXiv 5 May 2024.

## Builds on (most cited first, 30 of 38)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [Scene as Occupancy](https://mlsanity.com/paper/2306.02851/) (2023)
- [Just Go with the Flow: Self-Supervised Scene Flow Estimation](https://mlsanity.com/paper/1912.00497/) (2019)
- [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021)
- [Improving Tuning-Free Real Image Editing with Proximal Guidance](https://mlsanity.com/paper/2306.05414/) (2023)
- [Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting](https://mlsanity.com/paper/2302.13130/) (2023)
- [ALSO: Automotive Lidar Self-supervision by Occupancy estimation](https://mlsanity.com/paper/2212.05867/) (2022)
- [Occupancy Flow Fields for Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2203.03875/) (2022)
- [Dynamic 3D Scene Analysis by Point Cloud Accumulation](https://mlsanity.com/paper/2207.12394/) (2022)
- [A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmark](https://mlsanity.com/paper/2203.00131/) (2022)
- [Fast Neural Scene Flow](https://mlsanity.com/paper/2304.09121/) (2023)
- [Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision](https://mlsanity.com/paper/2303.00462/) (2023)
- [Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions](https://mlsanity.com/paper/2206.04129/) (2022)
- [Rolling-Shutter-Aware Differential SfM and Image Rectification](https://mlsanity.com/paper/1903.03943/) (2019)
- [Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving](https://mlsanity.com/paper/2308.01471/) (2023)
- [Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation](https://mlsanity.com/paper/2207.02201/) (2022)
- [Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2303.09447/) (2023)
- [TransFusion: Multi-view Divergent Fusion for Medical Image Segmentation with Transformers](https://mlsanity.com/paper/2203.10726/) (2022)
- [DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Method](https://mlsanity.com/paper/2206.07163/) (2022)
- [PillarFlow: End-to-end Birds-eye-view Flow Estimation for Autonomous Driving](https://mlsanity.com/paper/2008.01179/) (2020)
- [Building Volumetric Beliefs for Dynamic Environments Exploiting Map-Based Moving Object Segmentation](https://mlsanity.com/paper/2307.08314/) (2023)
- [The Implicit Values of A Good Hand Shake: Handheld Multi-Frame Neural Depth Refinement](https://mlsanity.com/paper/2111.13738/) (2021)

## Built on by (most cited first, 3 of 3)

- [LUCAS: Layered Universal Codec Avatars](https://mlsanity.com/paper/2502.19739/) (2025)
- [Snapmoji: Instant Generation of Animatable Dual-Stylized Avatars](https://mlsanity.com/paper/2503.11978/) (2025)
- [Large Sign Language Models: Toward 3D American Sign Language Translation](https://mlsanity.com/paper/2511.08535/) (2025)

## Abstract (arXiv)

The perception of 3D motion of surrounding traffic participants is crucial for driving safety. While existing works primarily focus on general large motions, we contend that the instantaneous detection and quantification of subtle motions is equally important as they indicate the nuances in driving behavior that may be safety critical, such as behaviors near a stop sign of parking positions. We delve into this under-explored task, examining its unique challenges and developing our solution, accompanied by a carefully designed benchmark. Specifically, due to the lack of correspondences between consecutive frames of sparse Lidar point clouds, static objects might appear to be moving - the so-called swimming effect. This intertwines with the true object motion, thereby posing ambiguity in accurate estimation, especially for subtle motions. To address this, we propose to leverage local occupancy completion of object point clouds to densify the shape cue, and mitigate the impact of swimming artifacts. The occupancy completion is learned in an end-to-end fashion together with the detection of moving objects and the estimation of their motion, instantaneously as soon as objects start to move. Extensive experiments demonstrate superior performance compared to standard 3D motion estimation approaches, particularly highlighting our method's specialized treatment of subtle motions.