# On Extensions of CLEVER: A Neural Network Robustness Evaluation Algorithm

arXiv: https://arxiv.org/abs/1810.08640 · HTML: https://mlsanity.com/paper/1810.08640/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2018.

## Built on by (most cited first, 5 of 5)

- [Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead](https://mlsanity.com/paper/2101.02559/) (2021)
- [Group-based Robustness: A General Framework for Customized Robustness in the Real World](https://mlsanity.com/paper/2306.16614/) (2023)
- [humancompatible.interconnect: Testing Properties of Repeated Uses of Interconnections of AI Systems](https://mlsanity.com/paper/2507.09626/) (2025)
- [Stochastic Sample Approximations of (Local) Moduli of Continuity](https://mlsanity.com/paper/2509.15368/) (2025)
- [Can Attack Difficulty Be Characterized Before Optimization? A Study of Pre-optimization Difficulty in Person-Vanishing Attacks](https://mlsanity.com/paper/2609.34501/) (2026)

## Abstract (arXiv)

CLEVER (Cross-Lipschitz Extreme Value for nEtwork Robustness) is an Extreme Value Theory (EVT) based robustness score for large-scale deep neural networks (DNNs). In this paper, we propose two extensions on this robustness score. First, we provide a new formal robustness guarantee for classifier functions that are twice differentiable. We apply extreme value theory on the new formal robustness guarantee and the estimated robustness is called second-order CLEVER score. Second, we discuss how to handle gradient masking, a common defensive technique, using CLEVER with Backward Pass Differentiable Approximation (BPDA). With BPDA applied, CLEVER can evaluate the intrinsic robustness of neural networks of a broader class -- networks with non-differentiable input transformations. We demonstrate the effectiveness of CLEVER with BPDA in experiments on a 121-layer Densenet model trained on the ImageNet dataset.