# Learning Representation for Anomaly Detection of Vehicle Trajectories

arXiv: https://arxiv.org/abs/2303.05000 · HTML: https://mlsanity.com/paper/2303.05000/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 24 of 24)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [Multimodal Motion Prediction with Stacked Transformers](https://mlsanity.com/paper/2103.11624/) (2021)
- [Generative Probabilistic Novelty Detection with Adversarial Autoencoders](https://mlsanity.com/paper/1807.02588/) (2018)
- [On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2201.05057/) (2022)
- [Anomaly Detection using One-Class Neural Networks](https://mlsanity.com/paper/1802.06360/) (2018)
- [Driver Anomaly Detection: A Dataset and Contrastive Learning Approach](https://mlsanity.com/paper/2009.14660/) (2020)
- [Online Anomaly Detection in Surveillance Videos with Asymptotic Bounds on False Alarm Rate](https://mlsanity.com/paper/2010.07110/) (2020)
- [Anomaly Detection in Multi-Agent Trajectories for Automated Driving](https://mlsanity.com/paper/2110.07922/) (2021)
- [Efficient Global Robustness Certification of Neural Networks via Interleaving Twin-Network Encoding](https://mlsanity.com/paper/2203.14141/) (2022)
- [TAE: A Semi-supervised Controllable Behavior-aware Trajectory Generator and Predictor](https://mlsanity.com/paper/2203.01261/) (2022)
- [Physics-Aware Safety-Assured Design of Hierarchical Neural Network based Planner](https://mlsanity.com/paper/2201.09140/) (2022)
- [Safety-driven Interactive Planning for Neural Network-based Lane Changing](https://mlsanity.com/paper/2201.09112/) (2022)
- [End-to-end Uncertainty-based Mitigation of Adversarial Attacks to Automated Lane Centering](https://mlsanity.com/paper/2103.00345/) (2021)
- [Securing Connected Vehicle Applications with an Efficient Dual Cyber-Physical Blockchain Framework](https://mlsanity.com/paper/2102.07690/) (2021)

## Built on by (most cited first, 6 of 6)

- [Empowering Autonomous Driving with Large Language Models: A Safety Perspective](https://mlsanity.com/paper/2312.00812/) (2023)
- [VT-Former: An Exploratory Study on Vehicle Trajectory Prediction for Highway Surveillance through Graph Isomorphism and Transformer](https://mlsanity.com/paper/2311.06623/) (2023)
- [Language-Enhanced Latent Representations for Out-of-Distribution Detection in Autonomous Driving](https://mlsanity.com/paper/2405.01691/) (2024)
- [VegaEdge: Edge AI Confluence Anomaly Detection for Real-Time Highway IoT-Applications](https://mlsanity.com/paper/2311.07880/) (2023)
- [Hybrid Video Anomaly Detection for Anomalous Scenarios in Autonomous Driving](https://mlsanity.com/paper/2406.06423/) (2024)
- [TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosis](https://mlsanity.com/paper/2609.02540/) (2026)

## Abstract (arXiv)

Predicting the future trajectories of surrounding vehicles based on their history trajectories is a critical task in autonomous driving. However, when small crafted perturbations are introduced to those history trajectories, the resulting anomalous (or adversarial) trajectories can significantly mislead the future trajectory prediction module of the ego vehicle, which may result in unsafe planning and even fatal accidents. Therefore, it is of great importance to detect such anomalous trajectories of the surrounding vehicles for system safety, but few works have addressed this issue. In this work, we propose two novel methods for learning effective and efficient representations for online anomaly detection of vehicle trajectories. Different from general time-series anomaly detection, anomalous vehicle trajectory detection deals with much richer contexts on the road and fewer observable patterns on the anomalous trajectories themselves. To address these challenges, our methods exploit contrastive learning techniques and trajectory semantics to capture the patterns underlying the driving scenarios for effective anomaly detection under supervised and unsupervised settings, respectively. We conduct extensive experiments to demonstrate that our supervised method based on contrastive learning and unsupervised method based on reconstruction with semantic latent space can significantly improve the performance of anomalous trajectory detection in their corresponding settings over various baseline methods. We also demonstrate our methods' generalization ability to detect unseen patterns of anomalies.