# Large Scale Autonomous Driving Scenarios Clustering with Self-supervised Feature Extraction

arXiv: https://arxiv.org/abs/2103.16101 · HTML: https://mlsanity.com/paper/2103.16101/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 28 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 10 of 10)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [Video Representation Learning by Dense Predictive Coding](https://mlsanity.com/paper/1909.04656/) (2019)
- [Variational Autoencoder for Deep Learning of Images, Labels and Captions](https://mlsanity.com/paper/1609.08976/) (2016)
- [Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic](https://mlsanity.com/paper/1901.02705/) (2019)
- [An Unsupervised Random Forest Clustering Technique for Automatic Traffic Scenario Categorization](https://mlsanity.com/paper/2004.02121/) (2020)
- [Unsupervised Learning of Video Representations via Dense Trajectory Clustering](https://mlsanity.com/paper/2006.15731/) (2020)

## Built on by (most cited first, 4 of 4)

- [Expert-LaSTS: Expert-Knowledge Guided Latent Space for Traffic Scenarios](https://mlsanity.com/paper/2207.09120/) (2022)
- [ExAgt: Expert-guided Augmentation for Representation Learning of Traffic Scenarios](https://mlsanity.com/paper/2207.08609/) (2022)
- [Deep Occupancy-Predictive Representations for Autonomous Driving](https://mlsanity.com/paper/2303.04218/) (2023)
- [Behavior-Centric Extraction of Scenarios from Highway Traffic Data and their Domain-Knowledge-Guided Clustering using CVQ-VAE](https://mlsanity.com/paper/2603.16964/) (2026)

## Abstract (arXiv)

The clustering of autonomous driving scenario data can substantially benefit the autonomous driving validation and simulation systems by improving the simulation tests' completeness and fidelity. This article proposes a comprehensive data clustering framework for a large set of vehicle driving data. Existing algorithms utilize handcrafted features whose quality relies on the judgments of human experts. Additionally, the related feature compression methods are not scalable for a large data-set. Our approach thoroughly considers the traffic elements, including both in-traffic agent objects and map information. Meanwhile, we proposed a self-supervised deep learning approach for spatial and temporal feature extraction to avoid biased data representation. With the newly designed driving data clustering evaluation metrics based on data-augmentation, the accuracy assessment does not require a human-labeled data-set, which is subject to human bias. Via such unprejudiced evaluation metrics, we have shown our approach surpasses the existing methods that rely on handcrafted feature extractions.