# VM-MODNet: Vehicle Motion aware Moving Object Detection for Autonomous Driving

arXiv: https://arxiv.org/abs/2104.10985 · HTML: https://mlsanity.com/paper/2104.10985/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Lookahead Optimizer: k steps forward, 1 step back](https://mlsanity.com/paper/1907.08610/) (2019)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [Vehicle Detection from 3D Lidar Using Fully Convolutional Network](https://mlsanity.com/paper/1608.07916/) (2016)
- [FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos](https://mlsanity.com/paper/1701.05384/) (2017)
- [CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth](https://mlsanity.com/paper/1904.02028/) (2019)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [3D Fully Convolutional Network for Vehicle Detection in Point Cloud](https://mlsanity.com/paper/1611.08069/) (2016)
- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [FuseMODNet: Real-Time Camera and LiDAR based Moving Object Detection for robust low-light Autonomous Driving](https://mlsanity.com/paper/1910.05395/) (2019)
- [FisheyeMODNet: Moving Object detection on Surround-view Cameras for Autonomous Driving](https://mlsanity.com/paper/1908.11789/) (2019)
- [Real-time Dynamic Object Detection for Autonomous Driving using Prior 3D-Maps](https://mlsanity.com/paper/1809.11036/) (2018)
- [RST-MODNet: Real-time Spatio-temporal Moving Object Detection for Autonomous Driving](https://mlsanity.com/paper/1912.00438/) (2019)
- [Object Detection, Tracking, and Motion Segmentation for Object-level Video Segmentation](https://mlsanity.com/paper/1608.03066/) (2016)
- [Deep Semantic Classification for 3D LiDAR Data](https://mlsanity.com/paper/1706.08355/) (2017)

## Abstract (arXiv)

Moving object Detection (MOD) is a critical task in autonomous driving as moving agents around the ego-vehicle need to be accurately detected for safe trajectory planning. It also enables appearance agnostic detection of objects based on motion cues. There are geometric challenges like motion-parallax ambiguity which makes it a difficult problem. In this work, we aim to leverage the vehicle motion information and feed it into the model to have an adaptation mechanism based on ego-motion. The motivation is to enable the model to implicitly perform ego-motion compensation to improve performance. We convert the six degrees of freedom vehicle motion into a pixel-wise tensor which can be fed as input to the CNN model. The proposed model using Vehicle Motion Tensor (VMT) achieves an absolute improvement of 5.6% in mIoU over the baseline architecture. We also achieve state-of-the-art results on the public KITTI_MoSeg_Extended dataset even compared to methods which make use of LiDAR and additional input frames. Our model is also lightweight and runs at 85 fps on a TitanX GPU. Qualitative results are provided in https://youtu.be/ezbfjti-kTk.