# BEV-MODNet: Monocular Camera based Bird's Eye View Moving Object Detection for Autonomous Driving

arXiv: https://arxiv.org/abs/2107.04937 · HTML: https://mlsanity.com/paper/2107.04937/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 22 of its 28 references that are in the map.
- First posted on arXiv 11 Jul 2021.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 22 of 22)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Lookahead Optimizer: k steps forward, 1 step back](https://mlsanity.com/paper/1907.08610/) (2019)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [Orthographic Feature Transform for Monocular 3D Object Detection](https://mlsanity.com/paper/1811.08188/) (2018)
- [Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks](https://mlsanity.com/paper/2003.13402/) (2020)
- [IntentNet: Learning to Predict Intention from Raw Sensor Data](https://mlsanity.com/paper/2101.07907/) (2021)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos](https://mlsanity.com/paper/1701.05384/) (2017)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [MonoLayout: Amodal scene layout from a single image](https://mlsanity.com/paper/2002.08394/) (2020)
- [FuseMODNet: Real-Time Camera and LiDAR based Moving Object Detection for robust low-light Autonomous Driving](https://mlsanity.com/paper/1910.05395/) (2019)
- [Monocular Instance Motion Segmentation for Autonomous Driving: KITTI InstanceMotSeg Dataset and Multi-task Baseline](https://mlsanity.com/paper/2008.07008/) (2020)
- [FisheyeMODNet: Moving Object detection on Surround-view Cameras for Autonomous Driving](https://mlsanity.com/paper/1908.11789/) (2019)
- [Trained Trajectory based Automated Parking System using Visual SLAM on Surround View Cameras](https://mlsanity.com/paper/2001.02161/) (2020)
- [RST-MODNet: Real-time Spatio-temporal Moving Object Detection for Autonomous Driving](https://mlsanity.com/paper/1912.00438/) (2019)
- [Vision-based Driver Assistance Systems: Survey, Taxonomy and Advances](https://mlsanity.com/paper/2104.12583/) (2021)
- [Object Detection, Tracking, and Motion Segmentation for Object-level Video Segmentation](https://mlsanity.com/paper/1608.03066/) (2016)
- [Design of Real-time Semantic Segmentation Decoder for Automated Driving](https://mlsanity.com/paper/1901.06580/) (2019)

## Built on by (most cited first, 5 of 5)

- [Multi-camera Bird's Eye View Perception for Autonomous Driving](https://mlsanity.com/paper/2309.09080/) (2023)
- [DaF-BEVSeg: Distortion-aware Fisheye Camera based Bird's Eye View Segmentation with Occlusion Reasoning](https://mlsanity.com/paper/2404.06352/) (2024)
- [Interpretable and Flexible Target-Conditioned Neural Planners For Autonomous Vehicles](https://mlsanity.com/paper/2309.13485/) (2023)
- [Improved Single Camera BEV Perception Using Multi-Camera Training](https://mlsanity.com/paper/2409.02676/) (2024)
- [BEVMOSNet: Multimodal Fusion for BEV Moving Object Segmentation](https://mlsanity.com/paper/2503.03280/) (2025)

## Abstract (arXiv)

Detection of moving objects is a very important task in autonomous driving systems. After the perception phase, motion planning is typically performed in Bird's Eye View (BEV) space. This would require projection of objects detected on the image plane to top view BEV plane. Such a projection is prone to errors due to lack of depth information and noisy mapping in far away areas. CNNs can leverage the global context in the scene to project better. In this work, we explore end-to-end Moving Object Detection (MOD) on the BEV map directly using monocular images as input. To the best of our knowledge, such a dataset does not exist and we create an extended KITTI-raw dataset consisting of 12.9k images with annotations of moving object masks in BEV space for five classes. The dataset is intended to be used for class agnostic motion cue based object detection and classes are provided as meta-data for better tuning. We design and implement a two-stream RGB and optical flow fusion architecture which outputs motion segmentation directly in BEV space. We compare it with inverse perspective mapping of state-of-the-art motion segmentation predictions on the image plane. We observe a significant improvement of 13% in mIoU using the simple baseline implementation. This demonstrates the ability to directly learn motion segmentation output in BEV space. Qualitative results of our baseline and the dataset annotations can be found in https://sites.google.com/view/bev-modnet.