# TAO: Tolerance-Aware Optimistic Verification for Floating-Point Neural Networks

arXiv: https://arxiv.org/abs/2510.16028 · HTML: https://mlsanity.com/paper/2510.16028/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 62 references that are in the map.
- First posted on arXiv 15 Oct 2025.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Atom: Low-bit Quantization for Efficient and Accurate LLM Serving](https://mlsanity.com/paper/2310.19102/) (2023)
- [Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware](https://mlsanity.com/paper/1806.03287/) (2018)
- [Proof-of-Learning: Definitions and Practice](https://mlsanity.com/paper/2103.05633/) (2021)
- [SafetyNets: Verifiable Execution of Deep Neural Networks on an Untrusted Cloud](https://mlsanity.com/paper/1706.10268/) (2017)
- [Exploiting Verified Neural Networks via Floating Point Numerical Error](https://mlsanity.com/paper/2003.03021/) (2020)
- [I Know What You See: Power Side-Channel Attack on Convolutional Neural Network Accelerators](https://mlsanity.com/paper/1803.05847/) (2018)
- [Tools for Verifying Neural Models' Training Data](https://mlsanity.com/paper/2307.00682/) (2023)
- [opML: Optimistic Machine Learning on Blockchain](https://mlsanity.com/paper/2401.17555/) (2024)
- [A scalable verification solution for blockchains](https://mlsanity.com/paper/1908.04756/) (2019)
- [ezDPS: An Efficient and Zero-Knowledge Machine Learning Inference Pipeline](https://mlsanity.com/paper/2212.05428/) (2022)
- [3LegRace: Privacy-Preserving DNN Training over TEEs and GPUs](https://mlsanity.com/paper/2110.01229/) (2021)
- [Verde: Verification via Refereed Delegation for Machine Learning Programs](https://mlsanity.com/paper/2502.19405/) (2025)
- [Optimistic Verifiable Training by Controlling Hardware Nondeterminism](https://mlsanity.com/paper/2403.09603/) (2024)
- [Agatha: Smart Contract for DNN Computation](https://mlsanity.com/paper/2105.04919/) (2021)
- [No Soundness in the Real World: On the Challenges of the Verification of Deployed Neural Networks](https://mlsanity.com/paper/2506.01054/) (2025)
- [Proof-of-Learning with Incentive Security](https://mlsanity.com/paper/2404.09005/) (2024)
- [Revealing Floating-Point Accumulation Orders in Software/Hardware Implementations](https://mlsanity.com/paper/2411.00442/) (2024)
- [PTEENet: Post-Trained Early-Exit Neural Networks Augmentation for Inference Cost Optimization](https://mlsanity.com/paper/2501.02508/) (2025)
- [Proof of Diligence: Cryptoeconomic Security for Rollups](https://mlsanity.com/paper/2402.07241/) (2024)
- [SERENE: A Collusion Resilient Replication-based Verification Framework](https://mlsanity.com/paper/2404.11410/) (2024)
- [Proof of Sampling: A Nash Equilibrium-Based Verification Protocol for Decentralized Systems](https://mlsanity.com/paper/2405.00295/) (2024)

## Built on by (most cited first, 1 of 1)

- [IMMACULATE: A Practical LLM Auditing Framework via Verifiable Computation](https://mlsanity.com/paper/2602.22700/) (2026)

## Abstract (arXiv)

Neural networks increasingly run on hardware outside the user's control (cloud GPUs, inference marketplaces). Yet ML-as-a-Service reveals little about what actually ran or whether returned outputs faithfully reflect the intended inputs. Users lack recourse against service downgrades (model swaps, quantization, graph rewrites, or discrepancies like altered ad embeddings). Verifying outputs is hard because floating-point(FP) execution on heterogeneous accelerators is inherently nondeterministic. Existing approaches are either impractical for real FP neural networks or reintroduce vendor trust. We present TAO: a Tolerance Aware Optimistic verification protocol that accepts outputs within principled operator-level acceptance regions rather than requiring bitwise equality. TAO combines two error models: (i) sound per-operator IEEE-754 worst-case bounds and (ii) tight empirical percentile profiles calibrated across hardware. Discrepancies trigger a Merkle-anchored, threshold-guided dispute game that recursively partitions the computation graph until one operator remains, where adjudication reduces to a lightweight theoretical-bound check or a small honest-majority vote against empirical thresholds. Unchallenged results finalize after a challenge window, without requiring trusted hardware or deterministic kernels. We implement TAO as a PyTorch-compatible runtime and a contract layer currently deployed on Ethereum Holesky testnet. The runtime instruments graphs, computes per-operator bounds, and runs unmodified vendor kernels in FP32 with negligible overhead (0.3% on Qwen3-8B). Across CNNs, Transformers and diffusion models on A100, H100, RTX6000, RTX4090, empirical thresholds are 102-103 times tighter than theoretical bounds, and bound-aware adversarial attacks achieve 0% success. Together, TAO reconciles scalability with verifiability for real-world heterogeneous ML compute.