# Metamorphic Testing of a Deep Learning based Forecaster

arXiv: https://arxiv.org/abs/1907.06632 · HTML: https://mlsanity.com/paper/1907.06632/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2019.

## Built on by (most cited first, 3 of 3)

- [Testing Framework for Black-box AI Models](https://mlsanity.com/paper/2102.06166/) (2021)
- [HOMRS: High Order Metamorphic Relations Selector for Deep Neural Networks](https://mlsanity.com/paper/2107.04863/) (2021)
- [Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning](https://mlsanity.com/paper/2606.06056/) (2026)

## Abstract (arXiv)

In this paper, we present the Metamorphic Testing of an in-use deep learning based forecasting application. The application looks at the past data of system characteristics (e.g. `memory allocation') to predict outages in the future. We focus on two statistical / machine learning based components - a) detection of co-relation between system characteristics and b) estimating the future value of a system characteristic using an LSTM (a deep learning architecture). In total, 19 Metamorphic Relations have been developed and we provide proofs & algorithms where applicable. We evaluated our method through two settings. In the first, we executed the relations on the actual application and uncovered 8 issues not known before. Second, we generated hypothetical bugs, through Mutation Testing, on a reference implementation of the LSTM based forecaster and found that 65.9% of the bugs were caught through the relations.