# Automatic Construction of Multi-layer Perceptron Network from Streaming Examples

arXiv: https://arxiv.org/abs/1910.03437 · HTML: https://mlsanity.com/paper/1910.03437/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2019.

## Built on by (most cited first, 11 of 11)

- [Unsupervised Continual Learning via Self-Adaptive Deep Clustering Approach](https://mlsanity.com/paper/2106.14563/) (2021)
- [Unsupervised Continual Learning in Streaming Environments](https://mlsanity.com/paper/2109.09282/) (2021)
- [Continual Learning via Inter-Task Synaptic Mapping](https://mlsanity.com/paper/2106.13954/) (2021)
- [Automatic Online Multi-Source Domain Adaptation](https://mlsanity.com/paper/2109.01996/) (2021)
- [Augmented Bilinear Network for Incremental Multi-Stock Time-Series Classification](https://mlsanity.com/paper/2207.11577/) (2022)
- [Autonomous Cross Domain Adaptation under Extreme Label Scarcity](https://mlsanity.com/paper/2209.01548/) (2022)
- [Autonomous Deep Quality Monitoring in Streaming Environments](https://mlsanity.com/paper/2106.13955/) (2021)
- [Scalable Teacher Forcing Network for Semi-Supervised Large Scale Data Streams](https://mlsanity.com/paper/2107.02943/) (2021)
- [A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspective](https://mlsanity.com/paper/2402.12627/) (2024)
- [Advances in Electron Microscopy with Deep Learning](https://mlsanity.com/paper/2101.01178/) (2021)
- [Dynamic Long-Term Time-Series Forecasting via Meta Transformer Networks](https://mlsanity.com/paper/2401.13968/) (2024)

## Abstract (arXiv)

Autonomous construction of deep neural network (DNNs) is desired for data streams because it potentially offers two advantages: proper model's capacity and quick reaction to drift and shift. While the self-organizing mechanism of DNNs remains an open issue, this task is even more challenging to be developed for standard multi-layer DNNs than that using the different-depth structures, because the addition of a new layer results in information loss of previously trained knowledge. A Neural Network with Dynamically Evolved Capacity (NADINE) is proposed in this paper. NADINE features a fully open structure where its network structure, depth and width, can be automatically evolved from scratch in an online manner and without the use of problem-specific thresholds. NADINE is structured under a standard MLP architecture and the catastrophic forgetting issue during the hidden layer addition phase is resolved using the proposal of soft-forgetting and adaptive memory methods. The advantage of NADINE, namely elastic structure and online learning trait, is numerically validated using nine data stream classification and regression problems where it demonstrates performance improvement over prominent algorithms in all problems. In addition, it is capable of dealing with data stream regression and classification problems equally well.