# Self-Supervised Velocity Estimation for Automotive Radar Object Detection Networks

arXiv: https://arxiv.org/abs/2207.03146 · HTML: https://mlsanity.com/paper/2207.03146/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 7 Jul 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PIXOR: Real-time 3D Object Detection from Point Clouds](https://mlsanity.com/paper/1902.06326/) (2019)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects](https://mlsanity.com/paper/2007.14366/) (2020)
- [CNN based Road User Detection using the 3D Radar Cube](https://mlsanity.com/paper/2004.12165/) (2020)
- [2D Car Detection in Radar Data with PointNets](https://mlsanity.com/paper/1904.08414/) (2019)
- [Multi-View Radar Semantic Segmentation](https://mlsanity.com/paper/2103.16214/) (2021)
- [Pointillism: Accurate 3D bounding box estimation with multi-radars](https://mlsanity.com/paper/2203.04440/) (2022)
- [Deep Radar Detector](https://mlsanity.com/paper/1906.12187/) (2019)
- [Radar Voxel Fusion for 3D Object Detection](https://mlsanity.com/paper/2106.14087/) (2021)
- [Off-the-shelf sensor vs. experimental radar -- How much resolution is necessary in automotive radar classification?](https://mlsanity.com/paper/2006.05485/) (2020)
- [DeepReflecs: Deep Learning for Automotive Object Classification with Radar Reflections](https://mlsanity.com/paper/2010.09273/) (2020)
- [Deep Learning on Radar Centric 3D Object Detection](https://mlsanity.com/paper/2003.00851/) (2020)

## Built on by (most cited first, 5 of 5)

- [Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey](https://mlsanity.com/paper/2305.04691/) (2023)
- [DeepFusion: A Robust and Modular 3D Object Detector for Lidars, Cameras and Radars](https://mlsanity.com/paper/2209.12729/) (2022)
- [A recurrent CNN for online object detection on raw radar frames](https://mlsanity.com/paper/2212.11172/) (2022)
- [Teacher-Student Knowledge Distillation for Radar Perception on Embedded Accelerators](https://mlsanity.com/paper/2303.07586/) (2023)
- [Group Regression for Query Based Object Detection and Tracking](https://mlsanity.com/paper/2308.14481/) (2023)

## Abstract (arXiv)

This paper presents a method to learn the Cartesian velocity of objects using an object detection network on automotive radar data. The proposed method is self-supervised in terms of generating its own training signal for the velocities. Labels are only required for single-frame, oriented bounding boxes (OBBs). Labels for the Cartesian velocities or contiguous sequences, which are expensive to obtain, are not required. The general idea is to pre-train an object detection network without velocities using single-frame OBB labels, and then exploit the network's OBB predictions on unlabelled data for velocity training. In detail, the network's OBB predictions of the unlabelled frames are updated to the timestamp of a labelled frame using the predicted velocities and the distances between the updated OBBs of the unlabelled frame and the OBB predictions of the labelled frame are used to generate a self-supervised training signal for the velocities. The detection network architecture is extended by a module to account for the temporal relation of multiple scans and a module to represent the radars' radial velocity measurements explicitly. A two-step approach of first training only OBB detection, followed by training OBB detection and velocities is used. Further, a pre-training with pseudo-labels generated from radar radial velocity measurements bootstraps the self-supervised method of this paper. Experiments on the publicly available nuScenes dataset show that the proposed method almost reaches the velocity estimation performance of a fully supervised training, but does not require expensive velocity labels. Furthermore, we outperform a baseline method which uses only radial velocity measurements as labels.