mlsanityData to 7 Oct 2026

On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling

arXiv 2206.14581 ↗First posted 27 Jun 2022cs.ET cs.AI cs.ARcited by 1

Not on the map. Why?

Abstract (arXiv)

Synaptic memory consolidation has been heralded as one of the key mechanisms for supporting continual learning in neuromorphic Artificial Intelligence (AI) systems. Here we report that a Fowler-Nordheim (FN) quantum-tunneling device can implement synaptic memory consolidation similar to what can be achieved by algorithmic consolidation models like the cascade and the elastic weight consolidation (EWC) models. The proposed FN-synapse not only stores the synaptic weight but also stores the synapse's historical usage statistic on the device itself. We also show that the operation of the FN-synapse is near-optimal in terms of the synaptic lifetime and we demonstrate that a network comprising FN-synapses outperforms a comparable EWC network for a small benchmark continual learning task. With an energy footprint of femtojoules per synaptic update, we believe that the proposed FN-synapse provides an ultra-energy-efficient approach for implementing both synaptic memory consolidation and persistent learning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 15 of its 52 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

15 of 52 references.

References · 15

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  3. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  4. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  5. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  6. 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
  7. 2018Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgettingcited by 127
  8. 2019Low-Memory Neural Network Training: A Technical Reportcited by 56
  9. 2019In-memory hyperdimensional computingcited by 47
  10. 2021Synaptic metaplasticity in binarized neural networkscited by 28
  11. 2017Continual Learning Through Synaptic Intelligencecited by 13
  12. 2020Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computingcited by 10
3 more
  1. 2021An Adaptive Synaptic Array using Fowler-Nordheim Dynamic Analog Memorycited by 2
  2. 2021SPoTKD: A Protocol for Symmetric Key Distribution over Public Channels Using Self-Powered Timekeeping Devicescited by 1
  3. 2021Palimpsest Memories Stored in Memristive Synapsescited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Estimation of Energy-dissipation Lower-bounds for Neuromorphic Learning-in-memorycited by 0