# Topological Uncertainty: Monitoring trained neural networks through persistence of activation graphs

arXiv: https://arxiv.org/abs/2105.04404 · HTML: https://mlsanity.com/paper/2105.04404/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 7 May 2021.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/1801.00553/) (2018)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [Learning Confidence for Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/1802.04865/) (2018)
- [An introduction to Topological Data Analysis: fundamental and practical aspects for data scientists](https://mlsanity.com/paper/1710.04019/) (2017)
- [PersLay: A Neural Network Layer for Persistence Diagrams and New Graph Topological Signatures](https://mlsanity.com/paper/1904.09378/) (2019)
- [Controlling Neural Level Sets](https://mlsanity.com/paper/1905.11911/) (2019)
- [Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology](https://mlsanity.com/paper/1812.09764/) (2018)
- [Geometry Helps to Compare Persistence Diagrams](https://mlsanity.com/paper/1606.03357/) (2016)
- [On Characterizing the Capacity of Neural Networks using Algebraic Topology](https://mlsanity.com/paper/1802.04443/) (2018)
- [Topological Approaches to Deep Learning](https://mlsanity.com/paper/1811.01122/) (2018)
- [Topological Data Analysis of Decision Boundaries with Application to Model Selection](https://mlsanity.com/paper/1805.09949/) (2018)
- [TopoMap: A 0-dimensional Homology Preserving Projection of High-Dimensional Data](https://mlsanity.com/paper/2009.01512/) (2020)
- [Adversary Detection in Neural Networks via Persistent Homology](https://mlsanity.com/paper/1711.10056/) (2017)
- [Marketplace for AI Models](https://mlsanity.com/paper/2003.01593/) (2020)

## Built on by (most cited first, 20 of 20)

- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Topological Detection of Trojaned Neural Networks](https://mlsanity.com/paper/2106.06469/) (2021)
- [Persformer: A Transformer Architecture for Topological Machine Learning](https://mlsanity.com/paper/2112.15210/) (2021)
- [Persistent Topological Features in Large Language Models](https://mlsanity.com/paper/2410.11042/) (2024)
- [Experimental Observations of the Topology of Convolutional Neural Network Activations](https://mlsanity.com/paper/2212.00222/) (2022)
- [Selecting which Dense Retriever to use for Zero-Shot Search](https://mlsanity.com/paper/2309.09403/) (2023)
- [Diffeomorphic interpolation for efficient persistence-based topological optimization](https://mlsanity.com/paper/2405.18820/) (2024)
- [Zeroth-Order Topological Insights into Iterative Magnitude Pruning](https://mlsanity.com/paper/2206.06563/) (2022)
- [MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks](https://mlsanity.com/paper/2305.13271/) (2023)
- [Addressing caveats of neural persistence with deep graph persistence](https://mlsanity.com/paper/2307.10865/) (2023)
- [Be Persistent: Towards a Unified Solution for Mitigating Shortcuts in Deep Learning](https://mlsanity.com/paper/2402.11237/) (2024)
- [Topology of Out-of-Distribution Examples in Deep Neural Networks](https://mlsanity.com/paper/2501.12522/) (2025)
- [Latent Space Topology Evolution in Multilayer Perceptrons](https://mlsanity.com/paper/2506.01569/) (2025)
- [Topological Uncertainty for Anomaly Detection in the Neural-network EoS Inference with Neutron Star Data](https://mlsanity.com/paper/2508.19683/) (2025)
- [Nonparametric Topological Layers in Neural Networks](https://mlsanity.com/paper/2111.14829/) (2021)
- [HAct: Out-of-Distribution Detection with Neural Net Activation Histograms](https://mlsanity.com/paper/2309.04837/) (2023)
- [Data Distribution Shifts in (Industrial) Federated Learning as a Privacy Issue](https://mlsanity.com/paper/2409.13875/) (2024)
- [Uncertainty of Network Topology with Applications to Out-of-Distribution Detection](https://mlsanity.com/paper/2511.18813/) (2025)
- [Understanding the geometry of deep learning with decision boundary volume](https://mlsanity.com/paper/2603.14768/) (2026)
- [Towards Scalable Persistence-Based Topological Optimization](https://mlsanity.com/paper/2605.10996/) (2026)

## Abstract (arXiv)

Although neural networks are capable of reaching astonishing performances on a wide variety of contexts, properly training networks on complicated tasks requires expertise and can be expensive from a computational perspective. In industrial applications, data coming from an open-world setting might widely differ from the benchmark datasets on which a network was trained. Being able to monitor the presence of such variations without retraining the network is of crucial importance. In this article, we develop a method to monitor trained neural networks based on the topological properties of their activation graphs. To each new observation, we assign a Topological Uncertainty, a score that aims to assess the reliability of the predictions by investigating the whole network instead of its final layer only, as typically done by practitioners. Our approach entirely works at a post-training level and does not require any assumption on the network architecture, optimization scheme, nor the use of data augmentation or auxiliary datasets; and can be faithfully applied on a large range of network architectures and data types. We showcase experimentally the potential of Topological Uncertainty in the context of trained network selection, Out-Of-Distribution detection, and shift-detection, both on synthetic and real datasets of images and graphs.