# Online Monitoring Framework for Automotive Time Series Data using JEPA Embeddings

arXiv: https://arxiv.org/abs/2602.09985 · HTML: https://mlsanity.com/paper/2602.09985/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 10 Feb 2026.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [VOS: Learning What You Don't Know by Virtual Outlier Synthesis](https://mlsanity.com/paper/2202.01197/) (2022)
- [Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learning](https://mlsanity.com/paper/2212.04994/) (2022)
- [Perceptual Grouping in Contrastive Vision-Language Models](https://mlsanity.com/paper/2210.09996/) (2022)
- [DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment](https://mlsanity.com/paper/2412.16334/) (2024)
- [FocalFormer3D : Focusing on Hard Instance for 3D Object Detection](https://mlsanity.com/paper/2308.04556/) (2023)
- [MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Features](https://mlsanity.com/paper/2307.12698/) (2023)
- [Reliable Trajectory Prediction and Uncertainty Quantification with Conditioned Diffusion Models](https://mlsanity.com/paper/2405.14384/) (2024)
- [T-JEPA: A Joint-Embedding Predictive Architecture for Trajectory Similarity Computation](https://mlsanity.com/paper/2406.12913/) (2024)
- [LS-VOS: Identifying Outliers in 3D Object Detections Using Latent Space Virtual Outlier Synthesis](https://mlsanity.com/paper/2310.00952/) (2023)
- [Novelty Detection and Analysis of Traffic Scenario Infrastructures in the Latent Space of a Vision Transformer-Based Triplet Autoencoder](https://mlsanity.com/paper/2105.01924/) (2021)
- [PEM: Perception Error Model for Virtual Testing of Autonomous Vehicles](https://mlsanity.com/paper/2302.11919/) (2023)
- [Hybrid Machine Learning Model with a Constrained Action Space for Trajectory Prediction](https://mlsanity.com/paper/2501.03666/) (2025)
- [Adaptive Neural Networks for Intelligent Data-Driven Development](https://mlsanity.com/paper/2502.10603/) (2025)
- [Assessing the Completeness of Traffic Scenario Categories for Automated Highway Driving Functions via Cluster-based Analysis](https://mlsanity.com/paper/2506.02599/) (2025)
- [ExAgt: Expert-guided Augmentation for Representation Learning of Traffic Scenarios](https://mlsanity.com/paper/2207.08609/) (2022)
- [Conditioning Latent-Space Clusters for Real-World Anomaly Classification](https://mlsanity.com/paper/2309.09676/) (2023)
- [Label-Free Model Failure Detection for Lidar-based Point Cloud Segmentation](https://mlsanity.com/paper/2407.14306/) (2024)
- [Inconsistency-based Active Learning for LiDAR Object Detection](https://mlsanity.com/paper/2505.00511/) (2025)

## Abstract (arXiv)

As autonomous vehicles are rolled out, measures must be taken to ensure their safe operation. In order to supervise a system that is already in operation, monitoring frameworks are frequently employed. These run continuously online in the background, supervising the system status and recording anomalies. This work proposes an online monitoring framework to detect anomalies in object state representations. Thereby, a key challenge is creating a framework for anomaly detection without anomaly labels, which are usually unavailable for unknown anomalies. To address this issue, this work applies a self-supervised embedding method to translate object data into a latent representation space. For this, a JEPA-based self-supervised prediction task is constructed, allowing training without anomaly labels and the creation of rich object embeddings. The resulting expressive JEPA embeddings serve as input for established anomaly detection methods, in order to identify anomalies within object state representations. This framework is particularly useful for applications in real-world environments, where new or unknown anomalies may occur during operation for which there are no labels available. Experiments performed on the publicly available, real-world nuScenes dataset illustrate the framework's capabilities.