# LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion

arXiv: https://arxiv.org/abs/2010.00731 · HTML: https://mlsanity.com/paper/2010.00731/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [Stepwise Goal-Driven Networks for Trajectory Prediction](https://mlsanity.com/paper/2103.14107/) (2021)
- [Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking](https://mlsanity.com/paper/2302.03802/) (2023)
- [CramNet: Camera-Radar Fusion with Ray-Constrained Cross-Attention for Robust 3D Object Detection](https://mlsanity.com/paper/2210.09267/) (2022)
- [TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Driving](https://mlsanity.com/paper/2303.09998/) (2023)
- [PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Map](https://mlsanity.com/paper/2204.10435/) (2022)
- [MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Data](https://mlsanity.com/paper/2104.10772/) (2021)
- [DeTra: A Unified Model for Object Detection and Trajectory Forecasting](https://mlsanity.com/paper/2406.04426/) (2024)
- [ADCNet: Learning from Raw Radar Data via Distillation](https://mlsanity.com/paper/2303.11420/) (2023)
- [Radar Voxel Fusion for 3D Object Detection](https://mlsanity.com/paper/2106.14087/) (2021)
- [NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving](https://mlsanity.com/paper/2209.14499/) (2022)
- [Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges](https://mlsanity.com/paper/2306.09304/) (2023)
- [Towards Robust Human Trajectory Prediction in Raw Videos](https://mlsanity.com/paper/2108.08259/) (2021)
- [Joint Perception and Prediction for Autonomous Driving: A Survey](https://mlsanity.com/paper/2412.14088/) (2024)
- [Investigating the Effect of Sensor Modalities in Multi-Sensor Detection-Prediction Models](https://mlsanity.com/paper/2101.03279/) (2021)
- [To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Software](https://mlsanity.com/paper/2508.20892/) (2025)
- [Timely Fusion of Surround Radar/Lidar for Object Detection in Autonomous Driving Systems](https://mlsanity.com/paper/2309.04806/) (2023)
- [Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle](https://mlsanity.com/paper/2309.06313/) (2023)

## Abstract (arXiv)

In this paper, we present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. Automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous radial velocity measurements. However, there are factors that make the fusion of lidar and radar information challenging, such as the relatively low angular resolution of radar measurements, their sparsity and the lack of exact time synchronization with lidar. To overcome these challenges, we propose an efficient spatio-temporal radar feature extraction scheme which achieves state-of-the-art performance on multiple large-scale datasets.Further, by incorporating radar information, we show a 52% reduction in prediction error for objects with high acceleration and a 16% reduction in prediction error for objects at longer range.