# Learning in Feedforward Neural Networks Accelerated by Transfer Entropy

arXiv: https://arxiv.org/abs/2104.14616 · HTML: https://mlsanity.com/paper/2104.14616/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 35 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 2 of 2)

- [Transfer entropy-based feedback improves performance in artificial neural networks](https://mlsanity.com/paper/1706.04265/) (2017)
- [Measures of Causality in Complex Datasets with application to financial data](https://mlsanity.com/paper/1401.1457/) (2014)

## Built on by (most cited first, 5 of 5)

- [Operator theory, kernels, and Feedforward Neural Networks](https://mlsanity.com/paper/2301.01327/) (2023)
- [Learning in Convolutional Neural Networks Accelerated by Transfer Entropy](https://mlsanity.com/paper/2404.02943/) (2024)
- [A deep learning approach to predict the number of k-barriers for intrusion detection over a circular region using wireless sensor networks](https://mlsanity.com/paper/2208.11887/) (2022)
- [Information Plane Analysis Visualization in Deep Learning via Transfer Entropy](https://mlsanity.com/paper/2404.01364/) (2024)
- [Transfer Entropy in Graph Convolutional Neural Networks](https://mlsanity.com/paper/2406.06632/) (2024)

## Abstract (arXiv)

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance.