mlsanityData to 7 Oct 2026

KAN-CL: Per-Knot Importance Regularization for Continual Learning with Kolmogorov-Arnold Networks

arXiv 2605.12306 ↗First posted 12 May 2026cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Catastrophic forgetting remains the central obstacle in continual learning (CL): parameters shared across tasks interfere with one another, and existing regularization methods such as EWC and SI apply uniform penalties without awareness of which input region a parameter serves. We propose KAN-CL, a continual learning framework that exploits the compact-support spline parameterization of Kolmogorov-Arnold Networks (KANs) to perform importance-weighted anchoring at per-knot granularity. Deployed as a classification head on a convolutional backbone with standard EWC regularization on the backbone (bbEWC) KAN-CL achieves forgetting reductions of 88% and 93% over a head-only KAN baseline on Split-CIFAR-10/5T and Split-CIFAR-100/10T respectively, while matching or exceeding the accuracy of all baselines on both benchmarks. We further provide a Neural Tangent Kernel (NTK) analysis showing that KAN's spline locality induces a structural rank deficit in the cross-task NTK, yielding a forgetting bound that holds even in the feature-learning regime. These results establish that combining an architecture with natural parameter locality (KAN head) with a complementary backbone regularizer (bbEWC) yields a compositional and principled approach to catastrophic forgetting.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

18 of 27 references.

References · 18

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2016Learning without Forgettingcited by 2,465
  3. 2016Progressive Neural Networkscited by 1,332
  4. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  5. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  6. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  7. 2019Orthogonal Gradient Descent for Continual Learningcited by 361
  8. 2018Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgettingcited by 127
  9. 2024KAN 2.0: Kolmogorov-Arnold Networks Meet Sciencecited by 116
  10. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 99
  11. 2024TKAN: Temporal Kolmogorov-Arnold Networkscited by 70
  12. 2020Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descentcited by 68
6 more
  1. 2024A Survey on Kolmogorov-Arnold Networkcited by 67
  2. 2024Kolmogorov-Arnold Network for Satellite Image Classification in Remote Sensingcited by 46
  3. 2024Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Taskscited by 39
  4. 2024Kolmogorov-Arnold Networks: A Critical Assessment of Claims, Performance, and Practical Viabilitycited by 23
  5. 2024PowerMLP: An Efficient Version of KANcited by 14
  6. 2024A preliminary study on continual learning in computer vision using Kolmogorov-Arnold Networkscited by 7

Built on by

Not cited yet.