# Quantifiable Uncertainty: A Stochastic Consensus Multi-Agent RAG Framework for Robust Malware Detection

arXiv: https://arxiv.org/abs/2605.08385 · HTML: https://mlsanity.com/paper/2605.08385/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 8 May 2026.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models](https://mlsanity.com/paper/1804.04637/) (2018)
- [Malware Detection by Eating a Whole EXE](https://mlsanity.com/paper/1710.09435/) (2017)
- [TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time](https://mlsanity.com/paper/1807.07838/) (2018)
- [Dos and Don'ts of Machine Learning in Computer Security](https://mlsanity.com/paper/2010.09470/) (2020)
- [Functionality-preserving Black-box Optimization of Adversarial Windows Malware](https://mlsanity.com/paper/2003.13526/) (2020)
- [Transcending Transcend: Revisiting Malware Classification in the Presence of Concept Drift](https://mlsanity.com/paper/2010.03856/) (2020)
- [SOREL-20M: A Large Scale Benchmark Dataset for Malicious PE Detection](https://mlsanity.com/paper/2012.07634/) (2020)
- [Android Malware Detection using Deep Learning on API Method Sequences](https://mlsanity.com/paper/1712.08996/) (2017)
- [Applying Graph Analysis for Unsupervised Fast Malware Fingerprinting](https://mlsanity.com/paper/2510.12811/) (2025)
- [AsmRAG: LLM-Driven Malware Detection by Retrieving Functionally Similar Assembly Code](https://mlsanity.com/paper/2604.23196/) (2026)

## Abstract (arXiv)

While contemporary deep learning malware detectors define a dominant defense paradigm, their sophistication also exposes them to novel structural evasion attacks, a limitation we attribute to their inherent inability to express epistemic uncertainty. To address this challenge, we present MAGMA, a Retrieval-Augmented Generation (RAG) framework that decouples malware analysis into semantic code retrieval and probabilistic verification. In contrast to monolithic classifiers, MAGMA employs a dual-stream embedding scheme over assembly and pseudo-code representations to isolate Decision-Critical Functions (DCFs) from the noise of dead code. We further introduce a Stochastic Consistency Ensemble, in which multiple instances of the same reasoning agent independently evaluate the retrieval set under non-deterministic sampling. From this ensemble, we derive two complementary metrics: Function Evidence Strength (FES), a weighted aggregation of retrieval confidence, and the Evidence Conflict Score (ECS), defined as the Shannon entropy of the ensemble's predictive distribution. We show that elevated ECS values serve as an effective proxy for structural ambiguity, enabling the system to implement a principled ``reject-option'' policy. Extensive evaluation demonstrates that MAGMA achieves a 98.4% detection rate, substantially exceeding existing solutions.