# Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations

arXiv: https://arxiv.org/abs/1712.05043 · HTML: https://mlsanity.com/paper/1712.05043/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2017.

## Built on by (most cited first, 4 of 4)

- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [Variable Division and Optimization for Constrained Multiobjective Portfolio Problems](https://mlsanity.com/paper/2101.08552/) (2021)
- [EBIC.JL -- an Efficient Implementation of Evolutionary Biclustering Algorithm in Julia](https://mlsanity.com/paper/2105.01196/) (2021)
- [Credit spread approximation and improvement using random forest regression](https://mlsanity.com/paper/2106.07358/) (2021)

## Abstract (arXiv)

Deep Learning (DL) aims at learning the meaningful representations. A meaningful representation refers to the one that gives rise to significant performance improvement of associated Machine Learning (ML) tasks by replacing the raw data as the input. However, optimal architecture design and model parameter estimation in DL algorithms are widely considered to be intractable. Evolutionary algorithms are much preferable for complex and non-convex problems due to its inherent characteristics of gradient-free and insensitivity to local optimum. In this paper, we propose a computationally economical algorithm for evolving unsupervised deep neural networks to efficiently learn meaningful representations, which is very suitable in the current Big Data era where sufficient labeled data for training is often expensive to acquire. In the proposed algorithm, finding an appropriate architecture and the initialized parameter values for a ML task at hand is modeled by one computational efficient gene encoding approach, which is employed to effectively model the task with a large number of parameters. In addition, a local search strategy is incorporated to facilitate the exploitation search for further improving the performance. Furthermore, a small proportion labeled data is utilized during evolution search to guarantee the learnt representations to be meaningful. The performance of the proposed algorithm has been thoroughly investigated over classification tasks. Specifically, error classification rate on MNIST with 1.15% is reached by the proposed algorithm consistently, which is a very promising result against state-of-the-art unsupervised DL algorithms.