mlsanityData to 7 Oct 2026

ExAIS: Executable AI Semantics

arXiv 2202.09868 ↗First posted 20 Feb 2022cs.AI cs.LGcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Neural networks can be regarded as a new programming paradigm, i.e., instead of building ever-more complex programs through (often informal) logical reasoning in the programmers' mind, complex 'AI' systems are built by optimising generic neural network models with big data. In this new paradigm, AI frameworks such as TensorFlow and PyTorch play a key role, which is as essential as the compiler for traditional programs. It is known that the lack of a proper semantics for programming languages (such as C), i.e., a correctness specification for compilers, has contributed to many problematic program behaviours and security issues. While it is in general hard to have a correctness specification for compilers due to the high complexity of programming languages and their rapid evolution, we have a unique opportunity to do it right this time for neural networks (which have a limited set of functions, and most of them have stable semantics). In this work, we report our effort on providing a correctness specification of neural network frameworks such as TensorFlow. We specify the semantics of almost all TensorFlow layers in the logical programming language Prolog. We demonstrate the usefulness of the semantics through two applications. One is a fuzzing engine for TensorFlow, which features a strong oracle and a systematic way of generating valid neural networks. The other is a model validation approach which enables consistent bug reporting for TensorFlow models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 9 of its 42 references that are in the map. First posted on arXiv 20 Feb 2022.

Builds on

9 of 42 references.

References · 9

  1. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  2. 2019Machine Learning Testing: Survey, Landscapes and Horizonscited by 213
  3. 2018Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontierscited by 117
  4. 2019Taxonomy of Real Faults in Deep Learning Systemscited by 90
  5. 2018Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Modelscited by 69
  6. 2018Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testingcited by 30
  7. 2021DocTer: Documentation Guided Fuzzing for Testing Deep Learning API Functionscited by 21
  8. 2017Developing Bug-Free Machine Learning Systems With Formal Mathematicscited by 13
  9. 2018Combinatorial Testing for Deep Learning Systemscited by 8

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Semantic-Based Neural Network Repaircited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Tensor Logic: The Language of AI
  2. 2026To Neuro-Symbolic Classification and Beyond by Compiling Description Logic Ontologies to Probabilistic Circuits
  3. 2023COOL: A Constraint Object-Oriented Logic Programming Language and its Neural-Symbolic Compilation System
  4. 2022Reasoning Like Program Executors
  5. 2024Symbol Correctness in Deep Neural Networks Containing Symbolic Layers
  6. 2023Scallop: A Language for Neurosymbolic Programming
  7. 2025Categorical Construction of Logically Verifiable Neural Architectures
  8. 2026From Natural Language to Executable Narsese: A Neuro-Symbolic Benchmark and Pipeline for Reasoning with NARS