# NasHD: Efficient ViT Architecture Performance Ranking using Hyperdimensional Computing

arXiv: https://arxiv.org/abs/2209.11356 · HTML: https://mlsanity.com/paper/2209.11356/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 23 Sep 2022.

## Builds on (most cited first, 21 of 21)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions](https://mlsanity.com/paper/2006.02903/) (2020)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [AutoFormer: Searching Transformers for Visual Recognition](https://mlsanity.com/paper/2107.00651/) (2021)
- [Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective](https://mlsanity.com/paper/2102.11535/) (2021)
- [FastReID: A Pytorch Toolbox for General Instance Re-identification](https://mlsanity.com/paper/2006.02631/) (2020)
- [A Survey on Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2008.10937/) (2020)
- [Classification using Hyperdimensional Computing: A Review](https://mlsanity.com/paper/2004.11204/) (2020)
- [Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap](https://mlsanity.com/paper/2008.01475/) (2020)
- [GroupFace: Learning Latent Groups and Constructing Group-based Representations for Face Recognition](https://mlsanity.com/paper/2005.10497/) (2020)
- [Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search](https://mlsanity.com/paper/2101.11342/) (2021)
- [Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network](https://mlsanity.com/paper/2001.06268/) (2020)
- [Lightweight Multi-Branch Network for Person Re-Identification](https://mlsanity.com/paper/2101.10774/) (2021)
- [One-shot Learning for iEEG Seizure Detection Using End-to-end Binary Operations: Local Binary Patterns with Hyperdimensional Computing](https://mlsanity.com/paper/1809.01926/) (2018)
- [A Strong Baseline for Vehicle Re-Identification](https://mlsanity.com/paper/2104.10850/) (2021)
- [UFO: Unified Feature Optimization](https://mlsanity.com/paper/2207.10341/) (2022)
- [MoleHD: Ultra-Low-Cost Drug Discovery using Hyperdimensional Computing](https://mlsanity.com/paper/2106.02894/) (2021)
- [Automated Architecture Search for Brain-inspired Hyperdimensional Computing](https://mlsanity.com/paper/2202.05827/) (2022)
- [Vision Transformer with Convolutions Architecture Search](https://mlsanity.com/paper/2203.10435/) (2022)

## Abstract (arXiv)

Neural Architecture Search (NAS) is an automated architecture engineering method for deep learning design automation, which serves as an alternative to the manual and error-prone process of model development, selection, evaluation and performance estimation. However, one major obstacle of NAS is the extremely demanding computation resource requirements and time-consuming iterations particularly when the dataset scales. In this paper, targeting at the emerging vision transformer (ViT), we present NasHD, a hyperdimensional computing based supervised learning model to rank the performance given the architectures and configurations. Different from other learning based methods, NasHD is faster thanks to the high parallel processing of HDC architecture. We also evaluated two HDC encoding schemes: Gram-based and Record-based of NasHD on their performance and efficiency. On the VIMER-UFO benchmark dataset of 8 applications from a diverse range of domains, NasHD Record can rank the performance of nearly 100K vision transformer models with about 1 minute while still achieving comparable results with sophisticated models.