# Uncovering Zero-Shot Generalization Gaps in Time-Series Foundation Models Using Real-World Videos

arXiv: https://arxiv.org/abs/2509.26347 · HTML: https://mlsanity.com/paper/2509.26347/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [Large Language Models are Few-Shot Health Learners](https://mlsanity.com/paper/2305.15525/) (2023)
- [Structure Discovery in Nonparametric Regression through Compositional Kernel Search](https://mlsanity.com/paper/1302.4922/) (2013)
- [Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data](https://mlsanity.com/paper/2407.14985/) (2024)
- [ChatGPT Informed Graph Neural Network for Stock Movement Prediction](https://mlsanity.com/paper/2306.03763/) (2023)
- [Building Transportation Foundation Model via Generative Graph Transformer](https://mlsanity.com/paper/2305.14826/) (2023)
- [Visual Physics: Discovering Physical Laws from Videos](https://mlsanity.com/paper/1911.11893/) (2019)

## Built on by (most cited first, 1 of 1)

- [RMISC: A Large-scale Real-world Multivariate Corpus for Time Series Foundation Models](https://mlsanity.com/paper/2607.06504/) (2026)

## Abstract (arXiv)

Recent research on time-series foundation models (TSFMs) has underscored the scarcity of real-world data, often supplemented with synthetic sources in existing datasets, whose generalizability remains however debated. As such, in this work, we propose a novel benchmarking approach: in particular, we aim at building a curated dataset reflecting real world physical temporal dynamics, extracting temporal signals from real-world videos using optical flow. As such, we introduce REAL-V-TSFM, a novel dataset designed to capture rich and diverse time series derived from real-world videos. Experimental results on state-of-the-art TSFMs under zero-shot forecasting show that, despite strong performance on conventional benchmarks, these models exhibit performance degradation on the proposed dataset, suggesting limited generalizability to novel datasets. These findings underscore the need for novel approaches to acquiring time series data and highlight the lack of universality in recent TSFMs, while further validating the effectiveness of our video-based time series data extraction pipeline.