mlsanityData to 7 Oct 2026

Validity Is What You Need

arXiv 2510.27628 ↗First posted 31 Oct 2025cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

While AI agents have long been discussed and studied in computer science, today's Agentic AI systems are something new. We consider other definitions of Agentic AI and propose a new realist definition. Agentic AI is a software delivery mechanism, comparable to software as a service (SaaS), which puts an application to work autonomously in a complex enterprise setting. Recent advances in large language models (LLMs) as foundation models have driven excitement in Agentic AI. We note, however, that Agentic AI systems are primarily applications, not foundations, and so their success depends on validation by end users and principal stakeholders. The tools and techniques needed by the principal users to validate their applications are quite different from the tools and techniques used to evaluate foundation models. Ironically, with good validation measures in place, in many cases the foundation models can be replaced with much simpler, faster, and more interpretable models that handle core logic. When it comes to Agentic AI, validity is what you need. LLMs are one option that might achieve it.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 30 of its 46 references that are in the map. First posted on arXiv 31 Oct 2025.

Builds on

30 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Small Language Models are the Future of Agentic AI2 Jun
  2. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  3. 2023Harms from Increasingly Agentic Algorithmic Systems20 Feb

Other references · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  7. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  8. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  9. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  10. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  11. 2023Textbooks Are All You Needcited by 561
  12. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
15 more
  1. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  2. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  3. 2023AI Alignment: A Comprehensive Surveycited by 256
  4. 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 142
  5. 2024ARC Prize 2024: Technical Reportcited by 103
  6. 2025Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivitycited by 93
  7. 2022From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectorscited by 82
  8. 2024Exploiting LLM Quantizationcited by 61
  9. 2018Incomplete Contracting and AI Alignmentcited by 60
  10. 2024Large Language Model Safety: A Holistic Surveycited by 56
  11. 2025LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?cited by 52
  12. 2022Dislocated Accountabilities in the AI Supply Chain: Modularity and Developers' Notions of Responsibilitycited by 33
  13. 2024A Survey of Useful LLM Evaluationcited by 28
  14. 2025No Free Lunch with Guardrailscited by 10
  15. 2023Designing Fiduciary Artificial Intelligencecited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI
  2. 2026Beyond Component Testing: Validating Agentic AI Systems
  3. 2026Measurement Without Validity: The Compounding Reliability Problem in Agentic AI Evaluation
  4. 2026From Obligation to Specification: A Survey on Validating EU AI Act Requirements in RE
  5. 2026Agentic AI in Industry: Adoption Level and Deployment Barriers
  6. 2026Agent-as-a-Judge
  7. 2026Trustworthy Agentic AI: Failure Modes, Mitigation Strategies, and a Lifecycle Framework for Autonomous LLM Systems
  8. 2026Reproducible, Explainable, and Effective Evaluations of Agentic AI for Software Engineering