# Uncovering the Origins of Instability in Dynamical Systems: How Attention Mechanism Can Help?

arXiv: https://arxiv.org/abs/2212.09641 · HTML: https://mlsanity.com/paper/2212.09641/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 19 Dec 2022.

## Builds on (most cited first, 11 of 11)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Understanding Attention and Generalization in Graph Neural Networks](https://mlsanity.com/paper/1905.02850/) (2019)
- [Stability Properties of Graph Neural Networks](https://mlsanity.com/paper/1905.04497/) (2019)
- [LayoutGMN: Neural Graph Matching for Structural Layout Similarity](https://mlsanity.com/paper/2012.06547/) (2020)
- [Memory in network flows and its effects on spreading dynamics and community detection](https://mlsanity.com/paper/1305.4807/) (2013)
- [Controllability of Brain Networks](https://mlsanity.com/paper/1406.5197/) (2014)
- [SSGCNet: A Sparse Spectra Graph Convolutional Network for Epileptic EEG Signal Classification](https://mlsanity.com/paper/2203.12910/) (2022)
- [Using higher-order Markov models to reveal flow-based communities in networks](https://mlsanity.com/paper/1601.03516/) (2016)
- [Asymmetrically interacting spreading dynamics on complex layered networks](https://mlsanity.com/paper/1405.1905/) (2014)
- [Optimizing Hybrid Spreading in Metapopulations](https://mlsanity.com/paper/1409.7291/) (2014)
- [Contrastive Embedding Distribution Refinement and Entropy-Aware Attention for 3D Point Cloud Classification](https://mlsanity.com/paper/2201.11388/) (2022)

## Abstract (arXiv)

The behavior of the network and its stability are governed by both dynamics of individual nodes as well as their topological interconnections. Attention mechanism as an integral part of neural network models was initially designed for natural language processing (NLP), and so far, has shown excellent performance in combining dynamics of individual nodes and the coupling strengths between them within a network. Despite undoubted impact of attention mechanism, it is not yet clear why some nodes of a network get higher attention weights. To come up with more explainable solutions, we tried to look at the problem from stability perspective. Based on stability theory, negative connections in a network can create feedback loops or other complex structures by allowing information to flow in the opposite direction. These structures play a critical role in the dynamics of a complex system and can contribute to abnormal synchronization, amplification, or suppression. We hypothesized that those nodes that are involved in organizing such structures can push the entire network into instability modes and therefore need higher attention during analysis. To test this hypothesis, attention mechanism along with spectral and topological stability analyses was performed on a real-world numerical problem, i.e., a linear Multi Input Multi Output state-space model of a piezoelectric tube actuator. The findings of our study suggest that the attention should be directed toward the collective behaviour of imbalanced structures and polarity-driven structural instabilities within the network. The results demonstrated that the nodes receiving more attention cause more instability in the system. Our study provides a proof of concept to understand why perturbing some nodes of a network may cause dramatic changes in the network dynamics.