mlsanityData to 7 Oct 2026

Design Principles for Lifelong Learning AI Accelerators

arXiv 2310.04467 ↗First posted 5 Oct 2023cs.LG cs.AI cs.SYcited by 9

Not on the map. Why?

Abstract (arXiv)

Lifelong learning - an agent's ability to learn throughout its lifetime - is a hallmark of biological learning systems and a central challenge for artificial intelligence (AI). The development of lifelong learning algorithms could lead to a range of novel AI applications, but this will also require the development of appropriate hardware accelerators, particularly if the models are to be deployed on edge platforms, which have strict size, weight, and power constraints. Here, we explore the design of lifelong learning AI accelerators that are intended for deployment in untethered environments. We identify key desirable capabilities for lifelong learning accelerators and highlight metrics to evaluate such accelerators. We then discuss current edge AI accelerators and explore the future design of lifelong learning accelerators, considering the role that different emerging technologies could play.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 37 of its 130 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

37 of 130 references.

References · 37

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  3. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  4. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  5. 2019Three scenarios for continual learningcited by 610
  6. 2014OpenML: networked science in machine learningcited by 523
  7. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  8. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  9. 2019MLPerf Inference Benchmarkcited by 195
  10. 2020Adversarial Continual Learningcited by 140
  11. 2020A Neural Dirichlet Process Mixture Model for Task-Free Continual Learningcited by 133
  12. 2018Don't forget, there is more than forgetting: new metrics for Continual Learningcited by 113
25 more
  1. 2020Photonics for artificial intelligence and neuromorphic computingcited by 111
  2. 2018Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilizationcited by 104
  3. 2019Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challengescited by 92
  4. 2020A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learningcited by 86
  5. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  6. 2022Continual evaluation for lifelong learning: Identifying the stability gapcited by 55
  7. 2020A Statistical Framework for Low-bitwidth Training of Deep Neural Networkscited by 51
  8. 2019Dissecting the Graphcore IPU Architecture via Microbenchmarkingcited by 47
  9. 2016Towards Verified Artificial Intelligencecited by 36
  10. 2022ReckOn: A 28nm Sub-mm2 Task-Agnostic Spiking Recurrent Neural Network Processor Enabling On-Chip Learning over Second-Long Timescalescited by 35
  11. 2020A Modern Primer on Processing in Memorycited by 30
  12. 2021Synaptic metaplasticity in binarized neural networkscited by 28
  13. 2022Spiking Neural Network Integrated Circuits: A Review of Trends and Future Directionscited by 27
  14. 2021Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applicationscited by 17
  15. 2022Practical Recommendations for Replay-based Continual Learning Methodscited by 16
  16. 2022Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensionscited by 14
  17. 2018Deep Positron: A Deep Neural Network Using the Posit Number Systemcited by 13
  18. 2022Lifelong Learning Metricscited by 13
  19. 2021In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processorcited by 11
  20. 2021GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descentcited by 9
  21. 2020End-to-End Memristive HTM System for Pattern Recognition and Sequence Predictioncited by 6
  22. 2020Memory-Latency-Accuracy Trade-offs for Continual Learning on a RISC-V Extreme-Edge Nodecited by 5
  23. 2021PLAM: a Posit Logarithm-Approximate Multipliercited by 3
  24. 2018Neuromorphic Architecture for the Hierarchical Temporal Memorycited by 2
  25. 2020Metaplasticity in Multistate Memristor Synaptic Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 1; 2026: 2.

9 papers.

  1. 2024Continual Learning and Catastrophic Forgettingcited by 73
  2. 2023Continual Learning: Applications and the Road Forwardcited by 59
  3. 2023Green Edge AI: A Contemporary Surveycited by 24
  4. 2026Position: Modular Memory is the Key to Continual Learning Agentscited by 7
  5. 2024Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learningcited by 6
  6. 2025Minion Gated Recurrent Unit for Continual Learningcited by 1
  7. 2026Learning to Remember, Learn, and Forget in Attention-Based Modelscited by 1
  8. 2024Lightweight, error-tolerant edge detection using memristor-enabled stochastic logicscited by 0
  9. 2024Emulating Brain-like Rapid Learning in Neuromorphic Edge Computingcited by 0