# Training Feedforward Neural Networks with Standard Logistic Activations is Feasible

arXiv: https://arxiv.org/abs/1710.01013 · HTML: https://mlsanity.com/paper/1710.01013/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2017.

## Built on by (most cited first, 2 of 2)

- [Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2109.14545/) (2021)
- [On the biological plausibility of orthogonal initialisation for solving gradient instability in deep neural networks](https://mlsanity.com/paper/2211.08408/) (2022)

## Abstract (arXiv)

Training feedforward neural networks with standard logistic activations is considered difficult because of the intrinsic properties of these sigmoidal functions. This work aims at showing that these networks can be trained to achieve generalization performance comparable to those based on hyperbolic tangent activations. The solution consists on applying a set of conditions in parameter initialization, which have been derived from the study of the properties of a single neuron from an information-theoretic perspective. The proposed initialization is validated through an extensive experimental analysis.