# Unlocking the potential of two-point cells for energy-efficient and resilient training of deep nets

arXiv: https://arxiv.org/abs/2211.01950 · HTML: https://mlsanity.com/paper/2211.01950/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 68 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science](https://mlsanity.com/paper/1707.04780/) (2017)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing](https://mlsanity.com/paper/1910.05316/) (2019)
- [Sparse GPU Kernels for Deep Learning](https://mlsanity.com/paper/2006.10901/) (2020)
- [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://mlsanity.com/paper/1810.11393/) (2018)
- [Towards deep learning with segregated dendrites](https://mlsanity.com/paper/1610.00161/) (2016)
- [A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)](https://mlsanity.com/paper/1708.04198/) (2017)
- [How Can We Be So Dense? The Benefits of Using Highly Sparse Representations](https://mlsanity.com/paper/1903.11257/) (2019)
- [The Sparse Manifold Transform](https://mlsanity.com/paper/1806.08887/) (2018)
- [Large-Scale Neuromorphic Spiking Array Processors: A quest to mimic the brain](https://mlsanity.com/paper/1805.08932/) (2018)
- [Neuromorphic Hardware In The Loop: Training a Deep Spiking Network on the BrainScaleS Wafer-Scale System](https://mlsanity.com/paper/1703.01909/) (2017)
- [The Dynamics of Functional Brain Networks: Integrated Network States during Cognitive Function](https://mlsanity.com/paper/1511.02976/) (2015)
- [Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks](https://mlsanity.com/paper/2112.13896/) (2021)
- [Lip-Reading Driven Deep Learning Approach for Speech Enhancement](https://mlsanity.com/paper/1808.00046/) (2018)
- [XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification](https://mlsanity.com/paper/1709.00572/) (2017)
- [Attentive Cross-modal Connections for Deep Multimodal Wearable-based Emotion Recognition](https://mlsanity.com/paper/2108.02241/) (2021)
- [Context-sensitive neocortical neurons transform the effectiveness and efficiency of neural information processing](https://mlsanity.com/paper/2207.07338/) (2022)
- [A neuromorphic hardware architecture using the Neural Engineering Framework for pattern recognition](https://mlsanity.com/paper/1507.05695/) (2015)

## Built on by (most cited first, 5 of 5)

- [Leveraging dendritic properties to advance machine learning and neuro-inspired computing](https://mlsanity.com/paper/2306.08007/) (2023)
- [An Overlooked Role of Context-Sensitive Dendrites](https://mlsanity.com/paper/2408.11019/) (2024)
- [Beyond Attention: Toward Machines with Intrinsic Higher Mental States](https://mlsanity.com/paper/2505.06257/) (2025)
- [Scalable Machines with Intrinsic Higher Mental-State Dynamics](https://mlsanity.com/paper/2603.13453/) (2026)
- [Single layer tiny Co4 outpaces GPT-2 and GPT-BERT](https://mlsanity.com/paper/2510.08404/) (2025)

## Abstract (arXiv)

Context-sensitive two-point layer 5 pyramidal cells (L5PCs) were discovered as long ago as 1999. However, the potential of this discovery to provide useful neural computation has yet to be demonstrated. Here we show for the first time how a transformative L5PCs-driven deep neural network (DNN), termed the multisensory cooperative computing (MCC) architecture, can effectively process large amounts of heterogeneous real-world audio-visual (AV) data, using far less energy compared to best available 'point' neuron-driven DNNs. A novel highly-distributed parallel implementation on a Xilinx UltraScale+ MPSoC device estimates energy savings up to 245759  ×  50000 μJ (i.e., 62% less than the baseline model in a semi-supervised learning setup) where a single synapse consumes 8e-5μJ. In a supervised learning setup, the energy-saving can potentially reach up to 1250x less (per feedforward transmission) than the baseline model. The significantly reduced neural activity in MCC leads to inherently fast learning and resilience against sudden neural damage. This remarkable performance in pilot experiments demonstrates the embodied neuromorphic intelligence of our proposed cooperative L5PC that receives input from diverse neighbouring neurons as context to amplify the transmission of most salient and relevant information for onward transmission, from overwhelmingly large multimodal information utilised at the early stages of on-chip training. Our proposed approach opens new cross-disciplinary avenues for future on-chip DNN training implementations and posits a radical shift in current neuromorphic computing paradigms.