mlsanityData to 7 Oct 2026

Laws of Learning Dynamics and the Core of Learners

arXiv 2602.05026 ↗First posted 4 Feb 2026cs.LG cs.AI cs.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

We formulate the fundamental laws governing learning dynamics, namely the conservation law and the decrease of total entropy. Within this framework, we introduce an entropy-based lifelong ensemble learning method. We evaluate its effectiveness by constructing an immunization mechanism to defend against transfer-based adversarial attacks on the CIFAR-10 dataset. Compared with a naive ensemble formed by simply averaging models specialized on clean and adversarial samples, the resulting logifold achieves higher accuracy in most test cases, with particularly large gains under strong perturbations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 62 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

41 of 62 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  10. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  11. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  12. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
29 more
  1. 2015Trust Region Policy Optimizationcited by 2,503
  2. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  5. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  6. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  9. 2020In Search of Lost Domain Generalizationcited by 1,042
  10. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  11. 2018Robustness May Be at Odds with Accuracycited by 873
  12. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  13. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  14. 2013Domain Generalization via Invariant Feature Representationcited by 622
  15. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  16. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  17. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
  18. 2015Neural Module Networkscited by 375
  19. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
  20. 2014The Loss Surfaces of Multilayer Networkscited by 278
  21. 2023Multimodal Deep Learningcited by 257
  22. 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 241
  23. 2015Conditional Computation in Neural Networks for faster modelscited by 166
  24. 2024On Information-Theoretic Measures of Predictive Uncertaintycited by 16
  25. 2021Two Coupled Rejection Metrics Can Tell Adversarial Examples Apartcited by 14
  26. 2019Playing it Safe: Adversarial Robustness with an Abstain Optioncited by 9
  27. 2023Stratified Adversarial Robustness with Rejectioncited by 4
  28. 2023Mixing Classifiers to Alleviate the Accuracy-Robustness Trade-Offcited by 2
  29. 2024A logifold structure on measure spacecited by 2

Built on by

Not cited yet.