mlsanityData to 7 Oct 2026

Optimisation Is Not What You Need

arXiv 2507.03045 ↗First posted 3 Jul 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The Artificial Intelligence field has focused on developing optimisation methods to solve multiple problems, specifically problems that we thought to be only solvable through cognition. The obtained results have been outstanding, being able to even surpass the Turing Test. However, we have found that these optimisation methods share some fundamental flaws that impede them to become a true artificial cognition. Specifically, the field have identified catastrophic forgetting as a fundamental problem to develop such cognition. This paper formally proves that this problem is inherent to optimisation methods, and as such it will always limit approaches that try to solve the Artificial General Intelligence problem as an optimisation problem. Additionally, it addresses the problem of overfitting and discuss about other smaller problems that optimisation methods pose. Finally, it empirically shows how world-modelling methods avoid suffering from either problem. As a conclusion, the field of Artificial Intelligence needs to look outside the machine learning field to find methods capable of developing an artificial cognition.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 64 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

26 of 64 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Progressive Neural Networkscited by 1,332
  3. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  4. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  5. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  6. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  7. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  8. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  9. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  10. 2024Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsalcited by 114
  11. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  12. 2018Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgettingcited by 46
14 more
  1. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  2. 2024Parameter Efficient Fine-tuning of Self-supervised ViTs without Catastrophic Forgettingcited by 17
  3. 2024MuseumMaker: Continual Style Customization without Catastrophic Forgettingcited by 10
  4. 2021Studying Catastrophic Forgetting in Neural Ranking Modelscited by 9
  5. 2025MalCL: Leveraging GAN-Based Generative Replay to Combat Catastrophic Forgetting in Malware Classificationcited by 8
  6. 2019Catastrophic forgetting: still a problem for DNNscited by 7
  7. 2024Preventing Catastrophic Forgetting through Memory Networks in Continuous Detectioncited by 7
  8. 2021On robustness of generative representations against catastrophic forgettingcited by 5
  9. 2024From Manifestations to Cognitive Architectures: a Scalable Frameworkcited by 4
  10. 2020Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidationcited by 3
  11. 2024Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue Systemcited by 3
  12. 2024Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learningcited by 3
  13. 2024Unsupervised Cognitioncited by 2
  14. 2025Beating Transformers using Synthetic Cognitioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
  2. 2025Forgetting is Everywhere
  3. 2025Merge and Conquer: Evolutionarily Optimizing AI for 2048
  4. 2025Beyond Statistical Learning: Exact Learning Is Essential for General Intelligence
  5. 2025Evolutionary System 2 Reasoning: An Empirical Proof
  6. 2024Artificial Intelligence without Restriction Surpassing Human Intelligence with Probability One: Theoretical Insight into Secrets of the Brain with AI Twins of the Brain
  7. 2024Untrained neural networks can demonstrate memorization-independent abstract reasoning
  8. 2025Mitigating Catastrophic Forgetting in Mathematical Reasoning Finetuning through Mixed Training