# Scheduled Knowledge Acquisition on Lightweight Vector Symbolic Architectures for Brain-Computer Interfaces

arXiv: https://arxiv.org/abs/2403.13844 · HTML: https://mlsanity.com/paper/2403.13844/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 27 of 27)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [A Comprehensive Overhaul of Feature Distillation](https://mlsanity.com/paper/1904.01866/) (2019)
- [Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons](https://mlsanity.com/paper/1811.03233/) (2018)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [Does Knowledge Distillation Really Work?](https://mlsanity.com/paper/2106.05945/) (2021)
- [Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification](https://mlsanity.com/paper/2001.01536/) (2020)
- [Focal and Global Knowledge Distillation for Detectors](https://mlsanity.com/paper/2111.11837/) (2021)
- [Online Knowledge Distillation with Diverse Peers](https://mlsanity.com/paper/1912.00350/) (2019)
- [When Do Curricula Work?](https://mlsanity.com/paper/2012.03107/) (2020)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [Classification using Hyperdimensional Computing: A Review](https://mlsanity.com/paper/2004.11204/) (2020)
- [Vector Symbolic Architectures as a Computing Framework for Emerging Hardware](https://mlsanity.com/paper/2106.05268/) (2021)
- [FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations](https://mlsanity.com/paper/2012.12206/) (2020)
- [Deep Generative Symbolic Regression](https://mlsanity.com/paper/2401.00282/) (2023)
- [Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation](https://mlsanity.com/paper/2006.14284/) (2020)
- [Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface](https://mlsanity.com/paper/1708.06578/) (2017)
- [EEG-based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and their Applications](https://mlsanity.com/paper/2001.11337/) (2020)
- [Knowledge Distillation with Feature Maps for Image Classification](https://mlsanity.com/paper/1812.00660/) (2018)
- [Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shifts](https://mlsanity.com/paper/2209.11233/) (2022)
- [Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition](https://mlsanity.com/paper/1907.06017/) (2019)
- [A Brain-Inspired Low-Dimensional Computing Classifier for Inference on Tiny Devices](https://mlsanity.com/paper/2203.04894/) (2022)
- [Exploring Embedding Methods in Binary Hyperdimensional Computing: A Case Study for Motor-Imagery based Brain-Computer Interfaces](https://mlsanity.com/paper/1812.05705/) (2018)
- [Navigating Memory Construction by Global Pseudo-Task Simulation for Continual Learning](https://mlsanity.com/paper/2210.08442/) (2022)
- [MetaLDC: Meta Learning of Low-Dimensional Computing Classifiers for Fast On-Device Adaption](https://mlsanity.com/paper/2302.12347/) (2023)

## Built on by (most cited first, 1 of 1)

- [Towards Vector Optimization on Low-Dimensional Vector Symbolic Architecture](https://mlsanity.com/paper/2502.14075/) (2025)

## Abstract (arXiv)

Brain-Computer interfaces (BCIs) are typically designed to be lightweight and responsive in real-time to provide users timely feedback. Classical feature engineering is computationally efficient but has low accuracy, whereas the recent neural networks (DNNs) improve accuracy but are computationally expensive and incur high latency. As a promising alternative, the low-dimensional computing (LDC) classifier based on vector symbolic architecture (VSA), achieves small model size yet higher accuracy than classical feature engineering methods. However, its accuracy still lags behind that of modern DNNs, making it challenging to process complex brain signals. To improve the accuracy of a small model, knowledge distillation is a popular method. However, maintaining a constant level of distillation between the teacher and student models may not be the best way for a growing student during its progressive learning stages. In this work, we propose a simple scheduled knowledge distillation method based on curriculum data order to enable the student to gradually build knowledge from the teacher model, controlled by an α scheduler. Meanwhile, we employ the LDC/VSA as the student model to enhance the on-device inference efficiency for tiny BCI devices that demand low latency. The empirical results have demonstrated that our approach achieves better tradeoff between accuracy and hardware efficiency compared to other methods.