# Risks of ignoring uncertainty propagation in AI-augmented security pipelines

arXiv: https://arxiv.org/abs/2407.14540 · HTML: https://mlsanity.com/paper/2407.14540/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 66 references that are in the map.
- First posted on arXiv 14 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learning](https://mlsanity.com/paper/2207.08281/) (2022)
- [Tensor Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2001.05313/) (2020)
- [Do Developers Update Their Library Dependencies? An Empirical Study on the Impact of Security Advisories on Library Migration](https://mlsanity.com/paper/1709.04621/) (2017)
- [DEAR: A Novel Deep Learning-based Approach for Automated Program Repair](https://mlsanity.com/paper/2205.01859/) (2022)
- [AIBugHunter: A Practical Tool for Predicting, Classifying and Repairing Software Vulnerabilities](https://mlsanity.com/paper/2305.16615/) (2023)
- [Automated Patch Assessment for Program Repair at Scale](https://mlsanity.com/paper/1909.13694/) (2019)
- [Harnessing Evolution for Multi-Hunk Program Repair](https://mlsanity.com/paper/1906.08903/) (2019)
- [SeqTrans: Automatic Vulnerability Fix via Sequence to Sequence Learning](https://mlsanity.com/paper/2010.10805/) (2020)
- ["False negative -- that one is going to kill you": Understanding Industry Perspectives of Static Analysis based Security Testing](https://mlsanity.com/paper/2307.16325/) (2023)
- [Automated Vulnerability Detection in Source Code Using Quantum Natural Language Processing](https://mlsanity.com/paper/2303.07525/) (2023)
- [Probability bound analysis: A novel approach for quantifying parameter uncertainty in decision-analytic modeling and cost-effectiveness analysis](https://mlsanity.com/paper/2011.10398/) (2020)

## Built on by (most cited first, 1 of 1)

- [Neurosymbolic Inference On Foundation Models For Remote Sensing Text-to-image Retrieval With Complex Queries](https://mlsanity.com/paper/2512.14102/) (2025)

## Abstract (arXiv)

The use of AI technologies is being integrated into the secure development of software-based systems, with an increasing trend of composing AI-based subsystems (with uncertain levels of performance) into automated pipelines. This presents a fundamental research challenge and seriously threatens safety-critical domains. Despite the existing knowledge about uncertainty in risk analysis, no previous work has estimated the uncertainty of AI-augmented systems given the propagation of errors in the pipeline. We provide the formal underpinnings for capturing uncertainty propagation, develop a simulator to quantify uncertainty, and evaluate the simulation of propagating errors with one case study. We discuss the generalizability of our approach and its limitations and present recommendations for evaluation policies concerning AI systems. Future work includes extending the approach by relaxing the remaining assumptions and by experimenting with a real system.