# Creating a Causally Grounded Rating Method for Assessing the Robustness of AI Models for Time-Series Forecasting

arXiv: https://arxiv.org/abs/2502.12226 · HTML: https://mlsanity.com/paper/2502.12226/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 56 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Causal Inference for Time series Analysis: Problems, Methods and Evaluation](https://mlsanity.com/paper/2102.05829/) (2021)
- [Adversarial Attacks on Deep Models for Financial Transaction Records](https://mlsanity.com/paper/2106.08361/) (2021)
- [Untargeted, Targeted and Universal Adversarial Attacks and Defenses on Time Series](https://mlsanity.com/paper/2101.05639/) (2021)
- [A Neural Mean Embedding Approach for Back-door and Front-door Adjustment](https://mlsanity.com/paper/2210.06610/) (2022)
- [Targeted Attacks on Timeseries Forecasting](https://mlsanity.com/paper/2301.11544/) (2023)
- [Rating Sentiment Analysis Systems for Bias through a Causal Lens](https://mlsanity.com/paper/2302.02038/) (2023)
- [Advances in Automatically Rating the Trustworthiness of Text Processing Services](https://mlsanity.com/paper/2302.09079/) (2023)
- [The Future of Spreadsheets in the Big Data Era](https://mlsanity.com/paper/1801.10231/) (2018)
- [The Effect of Human v/s Synthetic Test Data and Round-tripping on Assessment of Sentiment Analysis Systems for Bias](https://mlsanity.com/paper/2401.12985/) (2024)
- [Taking Over the Stock Market: Adversarial Perturbations Against Algorithmic Traders](https://mlsanity.com/paper/2010.09246/) (2020)

## Abstract (arXiv)

AI models, including both time-series-specific and general-purpose Foundation Models (FMs), have demonstrated strong potential in time-series forecasting across sectors like finance. However, these models are highly sensitive to input perturbations, which can lead to prediction errors and undermine trust among stakeholders, including investors and analysts. To address this challenge, we propose a causally grounded rating framework to systematically evaluate model robustness by analyzing statistical and confounding biases under various noisy and erroneous input scenarios. Our framework is applied to a large-scale experimental setup involving stock price data from multiple industries and evaluates both uni-modal and multi-modal models, including Vision Transformer-based (ViT) models and FMs. We introduce six types of input perturbations and twelve data distributions to assess model performance. Results indicate that multi-modal and time-series-specific FMs demonstrate greater robustness and accuracy compared to general-purpose models. Further, to validate our framework's usability, we conduct a user study showcasing time-series models' prediction errors along with our computed ratings. The study confirms that our ratings reduce the difficulty for users in comparing the robustness of different models. Our findings can help stakeholders understand model behaviors in terms of robustness and accuracy for better decision-making even without access to the model weights and training data, i.e., black-box settings.