# Interpretability in Safety-Critical FinancialTrading Systems

arXiv: https://arxiv.org/abs/2109.15112 · HTML: https://mlsanity.com/paper/2109.15112/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [Synthesizing the preferred inputs for neurons in neural networks via deep generator networks](https://mlsanity.com/paper/1605.09304/) (2016)
- [Object Detectors Emerge in Deep Scene CNNs](https://mlsanity.com/paper/1412.6856/) (2014)
- [Conditional Time Series Forecasting with Convolutional Neural Networks](https://mlsanity.com/paper/1703.04691/) (2017)
- [Sequential Neural Models with Stochastic Layers](https://mlsanity.com/paper/1605.07571/) (2016)
- [GluonTS: Probabilistic Time Series Models in Python](https://mlsanity.com/paper/1906.05264/) (2019)
- [Deep Learning in Asset Pricing](https://mlsanity.com/paper/1904.00745/) (2019)
- [Cryptocurrency Portfolio Management with Deep Reinforcement Learning](https://mlsanity.com/paper/1612.01277/) (2016)
- [Adversarial Attacks on Probabilistic Autoregressive Forecasting Models](https://mlsanity.com/paper/2003.03778/) (2020)

## Built on by (most cited first, 1 of 1)

- [Towards a Framework for Evaluating Explanations in Automated Fact Verification](https://mlsanity.com/paper/2403.20322/) (2024)

## Abstract (arXiv)

Sophisticated machine learning (ML) models to inform trading in the financial sector create problems of interpretability and risk management. Seemingly robust forecasting models may behave erroneously in out of distribution settings. In 2020, some of the world's most sophisticated quant hedge funds suffered losses as their ML models were first underhedged, and then overcompensated. We implement a gradient-based approach for precisely stress-testing how a trading model's forecasts can be manipulated, and their effects on downstream tasks at the trading execution level. We construct inputs -- whether in changes to sentiment or market variables -- that efficiently affect changes in the return distribution. In an industry-standard trading pipeline, we perturb model inputs for eight S&P 500 stocks. We find our approach discovers seemingly in-sample input settings that result in large negative shifts in return distributions. We provide the financial community with mechanisms to interpret ML forecasts in trading systems. For the security community, we provide a compelling application where studying ML robustness necessitates that one capture an end-to-end system's performance rather than study a ML model in isolation. Indeed, we show in our evaluation that errors in the forecasting model's predictions alone are not sufficient for trading decisions made based on these forecasts to yield a negative return.