mlsanityData to 14 Oct 2026

The Role of Rigor in Artificial Intelligence

arXiv 2607.03634 ↗First posted 19 May 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence (AI) has achieved extraordinary capabilities despite lacking many of the conceptual and scientific foundations associated with mature disciplines. Unlike traditional sciences, where reliable technology typically emerges from theoretical understanding, modern AI has progressed largely through performance-driven iteration and "alchemical" experimentation. This tension motivates a systematic analysis of AI through the lens of rigor. We introduce a three-part framework consisting of conceptual rigor (clarifying foundational concepts), epistemic rigor (establishing scientific understanding), and operational rigor (ensuring reliable performance and deployment). Using this framework, we analyze competing conceptions of intelligence and understanding, the strengths and limitations of the empirical approach to deep learning, the power and pitfalls of benchmarks, and the obstacles to theory development posed by modern AI systems. We argue that the distinctive trajectory of AI arises from how forms of rigor interact across paradigms, resulting in the primacy of operational rigor in modern deep learning. This perspective helps explain both AI's rapid advances and its persistent uncertainties, while clarifying the challenges involved in transforming AI into a mature science and reliable technology.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 57 of its 110 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

57 of 110 references.

References · 57

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2017Deep reinforcement learning from human preferencescited by 3,912
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  11. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  12. 2020Extracting Training Data from Large Language Modelscited by 2,189
38 more
  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  3. 2019Attention is not Explanationcited by 929
  4. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  5. 2017Deep Reinforcement Learning that Matterscited by 754
  6. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  7. 2018Recognition in Terra Incognitacited by 658
  8. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  9. 2019Attention is not not Explanationcited by 636
  10. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  11. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  12. 2020Artificial Intelligence, Values and Alignmentcited by 448
  13. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  14. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  15. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  16. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 288
  17. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  18. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
  19. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  20. 2021Accounting for Variance in Machine Learning Benchmarkscited by 201
  21. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  22. 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
  23. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  24. 2021The Benchmark Lotterycited by 136
  25. 2017No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysiscited by 129
  26. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 106
  27. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 96
  28. 2018Verisig: verifying safety properties of hybrid systems with neural network controllerscited by 84
  29. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 84
  30. 2022Meaning without reference in large language modelscited by 76
  31. 2021Introduction to Neural Network Verificationcited by 63
  32. 2025Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?cited by 34
  33. 2026There Will Be a Scientific Theory of Deep Learningcited by 29
  34. 2024Understanding Transformers via N-gram Statisticscited by 26
  35. 2018Verification for Machine Learning, Autonomy, and Neural Networks Surveycited by 24
  36. 2025Data Poisoning in Deep Learning: A Surveycited by 20
  37. 2024Position: Why We Must Rethink Empirical Research in Machine Learningcited by 19
  38. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17

50 most cited of 57 All 57 (JSON)

Built on by

Not cited yet.