# Peephole: Predicting Network Performance Before Training

arXiv: https://arxiv.org/abs/1712.03351 · HTML: https://mlsanity.com/paper/1712.03351/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2017.

## Built on by (most cited first, 23 of 23)

- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?](https://mlsanity.com/paper/2203.14577/) (2022)
- [Homogeneous Architecture Augmentation for Neural Predictor](https://mlsanity.com/paper/2107.13153/) (2021)
- [NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Prediction](https://mlsanity.com/paper/2211.08024/) (2022)
- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation](https://mlsanity.com/paper/2108.04541/) (2021)
- [Training-free Neural Architecture Search for RNNs and Transformers](https://mlsanity.com/paper/2306.00288/) (2023)
- [NAR-Former V2: Rethinking Transformer for Universal Neural Network Representation Learning](https://mlsanity.com/paper/2306.10792/) (2023)
- [MathNAS: If Blocks Have a Role in Mathematical Architecture Design](https://mlsanity.com/paper/2311.04943/) (2023)
- [NN-Former: Rethinking Graph Structure in Neural Architecture Representation](https://mlsanity.com/paper/2507.00880/) (2025)
- [Cascade Bagging for Accuracy Prediction with Few Training Samples](https://mlsanity.com/paper/2108.05613/) (2021)
- [DAS: Neural Architecture Search via Distinguishing Activation Score](https://mlsanity.com/paper/2212.12132/) (2022)
- [Neural Architecture Search: Two Constant Shared Weights Initialisations](https://mlsanity.com/paper/2302.04406/) (2023)
- [Brain-inspired Evolutionary Architectures for Spiking Neural Networks](https://mlsanity.com/paper/2309.05263/) (2023)
- [GPNAS: A Neural Network Architecture Search Framework Based on Graphical Predictor](https://mlsanity.com/paper/2103.11820/) (2021)
- [An Automated Question-Answering Framework Based on Evolution Algorithm](https://mlsanity.com/paper/2201.10797/) (2022)
- [Augmenting Novelty Search with a Surrogate Model to Engineer Meta-Diversity in Ensembles of Classifiers](https://mlsanity.com/paper/2201.12896/) (2022)
- [Efficient Automatic Machine Learning via Design Graphs](https://mlsanity.com/paper/2210.12257/) (2022)
- [PredNAS: A Universal and Sample Efficient Neural Architecture Search Framework](https://mlsanity.com/paper/2210.14460/) (2022)
- [Language Embedding Meets Dynamic Graph: A New Exploration for Neural Architecture Representation Learning](https://mlsanity.com/paper/2506.07735/) (2025)
- [Graph Lineages and Skeletal Graph Products](https://mlsanity.com/paper/2508.00197/) (2025)
- [From Regression to Inference: Meta-Learning Predictors for Neural Architecture Search](https://mlsanity.com/paper/2605.09290/) (2026)
- [DTA-GT: Direction- and Topology-Aware Graph Transformer for Neural Network Representation Learning](https://mlsanity.com/paper/2610.16822/) (2026)

## Abstract (arXiv)

The quest for performant networks has been a significant force that drives the advancements of deep learning in recent years. While rewarding, improving network design has never been an easy journey. The large design space combined with the tremendous cost required for network training poses a major obstacle to this endeavor. In this work, we propose a new approach to this problem, namely, predicting the performance of a network before training, based on its architecture. Specifically, we develop a unified way to encode individual layers into vectors and bring them together to form an integrated description via LSTM. Taking advantage of the recurrent network's strong expressive power, this method can reliably predict the performances of various network architectures. Our empirical studies showed that it not only achieved accurate predictions but also produced consistent rankings across datasets -- a key desideratum in performance prediction.