mlsanityData to 7 Oct 2026

LaneRCNN: Distributed Representations for Graph-Centric Motion Forecasting

arXiv 2101.06653 ↗First posted 17 Jan 2021cs.CV cs.ROcited by 112

Not on the map. Why?

Abstract (arXiv)

Forecasting the future behaviors of dynamic actors is an important task in many robotics applications such as self-driving. It is extremely challenging as actors have latent intentions and their trajectories are governed by complex interactions between the other actors, themselves, and the maps. In this paper, we propose LaneRCNN, a graph-centric motion forecasting model. Importantly, relying on a specially designed graph encoder, we learn a local lane graph representation per actor (LaneRoI) to encode its past motions and the local map topology. We further develop an interaction module which permits efficient message passing among local graph representations within a shared global lane graph. Moreover, we parameterize the output trajectories based on lane graphs, a more amenable prediction parameterization. Our LaneRCNN captures the actor-to-actor and the actor-to-map relations in a distributed and map-aware manner. We demonstrate the effectiveness of our approach on the large-scale Argoverse Motion Forecasting Benchmark. We achieve the 1st place on the leaderboard and significantly outperform previous best results.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). Builds on 51 of its 66 references that are in the map. First posted on arXiv 17 Jan 2021.

Builds on

51 of 66 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2017Mask R-CNNcited by 7,259
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2016Layer Normalizationcited by 5,579
  8. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  9. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  10. 2015Gated Graph Sequence Neural Networkscited by 792
  11. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  12. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
38 more
  1. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  2. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  3. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  4. 2019Graph U-Netscited by 489
  5. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  6. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  7. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  8. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  9. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  10. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  11. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  12. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  13. 2020V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predictioncited by 292
  14. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  15. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  16. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
  17. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  18. 2017Social Attention: Modeling Attention in Human Crowdscited by 185
  19. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  20. 2019Multiple Futures Predictioncited by 178
  21. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  22. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  23. 2019Multi-Agent Tensor Fusion for Contextual Trajectory Predictioncited by 131
  24. 2016Graph-Structured Representations for Visual Question Answeringcited by 123
  25. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  26. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  27. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  28. 2020PiP: Planning-informed Trajectory Prediction for Autonomous Drivingcited by 80
  29. 2020What-If Motion Prediction for Autonomous Drivingcited by 80
  30. 2019LanczosNet: Multi-Scale Deep Graph Convolutional Networkscited by 73
  31. 2020DSDNet: Deep Structured self-Driving Networkcited by 61
  32. 2020End-to-end Contextual Perception and Prediction with Interaction Transformercited by 54
  33. 2018How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Predictioncited by 46
  34. 2017Situation Recognition with Graph Neural Networkscited by 34
  35. 2017CAR-Net: Clairvoyant Attentive Recurrent Networkcited by 34
  36. 2019Discrete Residual Flow for Probabilistic Pedestrian Behavior Predictioncited by 34
  37. 2019Relational Action Forecastingcited by 27
  38. 2020Perceive, Attend, and Drive: Learning Spatial Attention for Safe Self-Drivingcited by 24

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 26; 2023: 34; 2024: 14; 2025: 19; 2026: 6.

112 papers.

Sort:

By area:

  1. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  2. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  3. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  4. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  5. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 112
  6. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  7. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  8. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
  9. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  10. 2023LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraintscited by 50
  11. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
  12. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 47
38 more
  1. 2023Leveraging Future Relationship Reasoning for Vehicle Trajectory Predictioncited by 47
  2. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  3. 2022FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphscited by 41
  4. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  5. 2023ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptationcited by 39
  6. 2023PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Drivingcited by 33
  7. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  8. 2022SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Drivingcited by 31
  9. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  10. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  11. 2023Dynamic Scenario Representation Learning for Motion Forecasting with Heterogeneous Graph Convolutional Recurrent Networkscited by 27
  12. 2022Path-Aware Graph Attention for HD Maps in Motion Predictioncited by 23
  13. 2022Bootstrap Motion Forecasting With Self-Consistent Constraintscited by 23
  14. 2022R-Pred: Two-Stage Motion Prediction Via Tube-Query Attention-Based Trajectory Refinementcited by 23
  15. 2023Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directionscited by 23
  16. 2022CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attentioncited by 21
  17. 2023Learning Cooperative Trajectory Representations for Motion Forecastingcited by 21
  18. 2021Trajectory Prediction with Graph-based Dual-scale Context Fusioncited by 19
  19. 2023SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networkscited by 18
  20. 2024VisionTrap: Vision-Augmented Trajectory Prediction Guided by Textual Descriptionscited by 18
  21. 2021Collaborative Uncertainty in Multi-Agent Trajectory Forecastingcited by 17
  22. 2023EDA: Evolving and Distinct Anchors for Multimodal Motion Predictioncited by 17
  23. 2023A Diffusion-Model of Joint Interactive Navigationcited by 16
  24. 2021DenseTNT: Waymo Open Dataset Motion Prediction Challenge 1st Place Solutioncited by 14
  25. 2025Co-MTP: A Cooperative Trajectory Prediction Framework with Multi-Temporal Fusion for Autonomous Drivingcited by 14
  26. 2022Uncertainty estimation for Cross-dataset performance in Trajectory predictioncited by 13
  27. 2022Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecastingcited by 13
  28. 2023MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequencescited by 13
  29. 2021Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Predictioncited by 12
  30. 2022Diverse Multiple Trajectory Prediction Using a Two-stage Prediction Network Trained with Lane Losscited by 12
  31. 2023Context-Aware Timewise VAEs for Real-Time Vehicle Trajectory Predictioncited by 12
  32. 2024MFTraj: Map-Free, Behavior-Driven Trajectory Prediction for Autonomous Drivingcited by 12
  33. 2021Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modelingcited by 11
  34. 2022KEMP: Keyframe-Based Hierarchical End-to-End Deep Model for Long-Term Trajectory Predictioncited by 11
  35. 2024Motion Forecasting in Continuous Drivingcited by 11
  36. 2021On complementing end-to-end human behavior predictors with planningcited by 10
  37. 2022Narrowing the Coordinate-frame Gap in Behavior Prediction Models: Distillation for Efficient and Accurate Scene-centric Motion Forecastingcited by 10
  38. 2022Towards Explainable Motion Prediction using Heterogeneous Graph Representationscited by 10

50 most cited of 112 All 112 (JSON)