# Randomly Weighted Neuromodulation in Neural Networks Facilitates Learning of Manifolds Common Across Tasks

arXiv: https://arxiv.org/abs/2401.02437 · HTML: https://mlsanity.com/paper/2401.02437/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 17 Nov 2023.

## Builds on (most cited first, 12 of 12)

- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers](https://mlsanity.com/paper/1811.04918/) (2018)
- [Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity](https://mlsanity.com/paper/1602.05897/) (2016)
- [A Survey on Learning to Hash](https://mlsanity.com/paper/1606.00185/) (2016)
- [Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity](https://mlsanity.com/paper/2002.10585/) (2020)
- [Deep Reinforcement Learning and its Neuroscientific Implications](https://mlsanity.com/paper/2007.03750/) (2020)
- [Introducing Neuromodulation in Deep Neural Networks to Learn Adaptive Behaviours](https://mlsanity.com/paper/1812.09113/) (2018)
- [For Manifold Learning, Deep Neural Networks can be Locality Sensitive Hash Functions](https://mlsanity.com/paper/2103.06875/) (2021)
- [An Insect-Inspired Randomly, Weighted Neural Network with Random Fourier Features For Neuro-Symbolic Relational Learning](https://mlsanity.com/paper/2109.06663/) (2021)
- [Representing Prior Knowledge Using Randomly, Weighted Feature Networks for Visual Relationship Detection](https://mlsanity.com/paper/2111.10686/) (2021)
- [Learning to Modulate Random Weights: Neuromodulation-inspired Neural Networks For Efficient Continual Learning](https://mlsanity.com/paper/2204.04297/) (2022)

## Abstract (arXiv)

Geometric Sensitive Hashing functions, a family of Local Sensitive Hashing functions, are neural network models that learn class-specific manifold geometry in supervised learning. However, given a set of supervised learning tasks, understanding the manifold geometries that can represent each task and the kinds of relationships between the tasks based on them has received little attention. We explore a formalization of this question by considering a generative process where each task is associated with a high-dimensional manifold, which can be done in brain-like models with neuromodulatory systems. Following this formulation, we define Task-specific Geometric Sensitive Hashing~(T-GSH) and show that a randomly weighted neural network with a neuromodulation system can realize this function.