# Understanding Spending Behavior: Recurrent Neural Network Explanation and Interpretation

arXiv: https://arxiv.org/abs/2109.11871 · HTML: https://mlsanity.com/paper/2109.11871/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 10 of 10)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [AI in Finance: Challenges, Techniques and Opportunities](https://mlsanity.com/paper/2107.09051/) (2021)
- [Understanding Hidden Memories of Recurrent Neural Networks](https://mlsanity.com/paper/1710.10777/) (2017)
- [Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics](https://mlsanity.com/paper/1906.10720/) (2019)
- [Interpreting recurrent neural networks behaviour via excitable network attractors](https://mlsanity.com/paper/1807.10478/) (2018)
- [Clustering in Recurrent Neural Networks for Micro-Segmentation using Spending Personality](https://mlsanity.com/paper/2109.09425/) (2021)
- [Churn Prediction with Sequential Data and Deep Neural Networks. A Comparative Analysis](https://mlsanity.com/paper/1909.11114/) (2019)

## Built on by (most cited first, 3 of 3)

- [A Comprehensive Review on Financial Explainable AI](https://mlsanity.com/paper/2309.11960/) (2023)
- [Can Interpretable Reinforcement Learning Manage Prosperity Your Way?](https://mlsanity.com/paper/2202.09064/) (2022)
- [Reinforcement Learning with Intrinsic Affinity for Personalized Prosperity Management](https://mlsanity.com/paper/2204.09218/) (2022)

## Abstract (arXiv)

Micro-segmentation of customers in the finance sector is a non-trivial task and has been an atypical omission from recent scientific literature. Where traditional segmentation classifies customers based on coarse features such as demographics, micro-segmentation depicts more nuanced differences between individuals, bringing forth several advantages including the potential for improved personalization in financial services. AI and representation learning offer a unique opportunity to solve the problem of micro-segmentation. Although ubiquitous in many industries, the proliferation of AI in sensitive industries such as finance has become contingent on the explainability of deep models. We had previously solved the micro-segmentation problem by extracting temporal features from the state space of a recurrent neural network (RNN). However, due to the inherent opacity of RNNs our solution lacked an explanation. In this study, we address this issue by extracting a symbolic explanation for our model and providing an interpretation of our temporal features. For the explanation, we use a linear regression model to reconstruct the features in the state space with high fidelity. We show that our linear regression coefficients have not only learned the rules used to recreate the features, but have also learned the relationships that were not directly evident in the raw data. Finally, we propose a novel method to interpret the dynamics of the state space by using the principles of inverse regression and dynamical systems to locate and label a set of attractors.