mlsanityData to 14 Oct 2026

VT-Former: An Exploratory Study on Vehicle Trajectory Prediction for Highway Surveillance through Graph Isomorphism and Transformer

arXiv 2311.06623 ↗First posted 11 Nov 2023cs.CV cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Enhancing roadway safety has become an essential computer vision focus area for Intelligent Transportation Systems (ITS). As a part of ITS, Vehicle Trajectory Prediction (VTP) aims to forecast a vehicle's future positions based on its past and current movements. VTP is a pivotal element for road safety, aiding in applications such as traffic management, accident prevention, work-zone safety, and energy optimization. While most works in this field focus on autonomous driving, with the growing number of surveillance cameras, another sub-field emerges for surveillance VTP with its own set of challenges. In this paper, we introduce VT-Former, a novel transformer-based VTP approach for highway safety and surveillance. In addition to utilizing transformers to capture long-range temporal patterns, a new Graph Attentive Tokenization (GAT) module has been proposed to capture intricate social interactions among vehicles. This study seeks to explore both the advantages and the limitations inherent in combining transformer architecture with graphs for VTP. Our investigation, conducted across three benchmark datasets from diverse surveillance viewpoints, showcases the State-of-the-Art (SotA) or comparable performance of VT-Former in predicting vehicle trajectories. This study underscores the potential of VT-Former and its architecture, opening new avenues for future research and exploration.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 44 of its 95 references that are in the map. First posted on arXiv 11 Nov 2023.

Builds on

44 of 95 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2017Graph Attention Networkscited by 8,923
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2018How Powerful are Graph Neural Networks?cited by 4,393
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  9. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  10. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  11. 2021Transformers in Vision: A Surveycited by 1,013
  12. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
32 more
  1. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  2. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  3. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 325
  4. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  5. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  6. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 113
  7. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 110
  8. 2019Graph Embedded Pose Clustering for Anomaly Detectioncited by 95
  9. 2018An LSTM Network for Highway Trajectory Predictioncited by 81
  10. 2019GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Drivingcited by 73
  11. 2020Multi-View Fusion of Sensor Data for Improved Perception and Prediction in Autonomous Drivingcited by 41
  12. 2020MultiXNet: Multiclass Multistage Multimodal Motion Predictioncited by 31
  13. 2022On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Drivingcited by 30
  14. 2022GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Modelcited by 26
  15. 2022Multi-Camera Multiple 3D Object Tracking on the Move for Autonomous Vehiclescited by 20
  16. 2021Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertaintycited by 19
  17. 2021SSAGCN: Social Soft Attention Graph Convolution Network for Pedestrian Trajectory Predictioncited by 19
  18. 2021MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Datacited by 18
  19. 2022Control-Aware Prediction Objectives for Autonomous Drivingcited by 18
  20. 2019A Review of Tracking, Prediction and Decision Making Methods for Autonomous Drivingcited by 17
  21. 2021DeepTrack: Lightweight Deep Learning for Vehicle Path Prediction in Highwayscited by 16
  22. 2023Surround-View Vision-based 3D Detection for Autonomous Driving: A Surveycited by 14
  23. 2023Transformer-Based Sensor Fusion for Autonomous Driving: A Surveycited by 14
  24. 2021NetTraj: A Network-based Vehicle Trajectory Prediction Model with Directional Representation and Spatiotemporal Attention Mechanismscited by 13
  25. 2020CARPe Posterum: A Convolutional Approach for Real-time Pedestrian Path Predictioncited by 11
  26. 2021Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPScited by 10
  27. 2020Improving Movement Predictions of Traffic Actors in Bird's-Eye View Models using GANs and Differentiable Trajectory Rasterizationcited by 9
  28. 2022Pishgu: Universal Path Prediction Network Architecture for Real-time Cyber-physical Edge Systemscited by 9
  29. 2018A Scalable Framework for Trajectory Predictioncited by 7
  30. 2023Learning Representation for Anomaly Detection of Vehicle Trajectoriescited by 6
  31. 2020Reconfigurable Cyber-Physical System for Critical Infrastructure Protection in Smart Cities via Smart Video-Surveillancecited by 4
  32. 2023Artificial Intelligence for Smart Transportationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 4; 2026: 3.

10 papers.

  1. 2025Multi-modal Knowledge Distillation-based Human Trajectory Forecastingcited by 10
  2. 2024Human-Centric Video Anomaly Detection Through Spatio-Temporal Pose Tokenization and Transformercited by 6
  3. 2025Shopformer: Transformer-Based Framework for Detecting Shoplifting via Human Posecited by 6
  4. 2023From Lab to Field: Real-World Evaluation of an AI-Driven Smart Video Solution to Enhance Community Safetycited by 5
  5. 2024Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Surveycited by 4
  6. 2025A Survey on Video Anomaly Detection via Deep Learning: Human, Vehicle, and Environmentcited by 4
  7. 2025MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generationcited by 2
  8. 2026Forecasting the Past: Gradient-Based Distribution Shift Detection in Trajectory Predictioncited by 0
  9. 2026EdgeVTP: Exploration of Latency-efficient Trajectory Prediction for Edge-based Embedded Vision Applicationscited by 0
  10. 2026Sparse Compositional Flow Matching by geometric assembly from motion primitivescited by 0