# Secret Scanner Agent: Extracting Secrets and Access Context from Unstructured Documents

arXiv: https://arxiv.org/abs/2607.09011 · HTML: https://mlsanity.com/paper/2607.09011/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 20 of 20)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation](https://mlsanity.com/paper/2201.05955/) (2022)
- [Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources](https://mlsanity.com/paper/2409.08239/) (2024)
- [DataGen: Unified Synthetic Dataset Generation via Large Language Models](https://mlsanity.com/paper/2406.18966/) (2024)
- [Interpreting the Curse of Dimensionality from Distance Concentration and Manifold Effect](https://mlsanity.com/paper/2401.00422/) (2023)
- [Machine Learning vs Deep Learning: The Generalization Problem](https://mlsanity.com/paper/2403.01621/) (2024)
- [UTNLP at SemEval-2022 Task 6: A Comparative Analysis of Sarcasm Detection Using Generative-based and Mutation-based Data Augmentation](https://mlsanity.com/paper/2204.08198/) (2022)
- [A Comparative Study of Software Secrets Reporting by Secret Detection Tools](https://mlsanity.com/paper/2307.00714/) (2023)
- [Detecting Hard-Coded Credentials in Software Repositories via LLMs](https://mlsanity.com/paper/2506.13090/) (2025)
- [Secret Leak Detection in Software Issue Reports using LLMs: A Comprehensive Evaluation](https://mlsanity.com/paper/2410.23657/) (2024)
- [You Have Been LaTeXpOsEd: A Systematic Analysis of Information Leakage in Preprint Archives Using Large Language Models](https://mlsanity.com/paper/2510.03761/) (2025)
- [Evaluating Large Language Models in detecting Secrets in Android Apps](https://mlsanity.com/paper/2510.18601/) (2025)

## Abstract (arXiv)

Exposed documents such as emails, chat threads, tickets, and incident notes routinely leak credentials, but during incident response a leaked secret is only half the story. Responders also need to identify the ``door'' the secret opens: the account, tenant, endpoint, database, cloud resource, or other system that the credential could allow an attacker to access. Traditional secret scanners rely on regular expressions or trained classifiers which work well on well-formatted code, yet they struggle when a credential is fragmented, reformatted, or far from the resource it unlocks, and they report the secret string without naming what it opens. We present Secret Scanner Agent (SSA), a multi-agent large-language-model system that extracts both the secret and its associated door, together with supporting evidence, from unstructured exposed documents. SSA pairs a detection agent that favors recall with a review agent that filters false positives and recovers missing context. Because real credential data is sensitive, we evaluate SSA on synthetic benchmarks we generated that span 23 secret types and multiple document formats, scored with a three-step pipeline of programmatic matching, an LLM judge, and human review. Across six models, multi-agent SSA improves extraction precision over a single-agent variant, with the largest gains on door extraction, by up to 16 percentage points. SSA matches a regular-expression scanner's precision while more than tripling its recall, and against thirteen security analysts it is more precise, recovers nearly twice as many secret--door pairs, and runs five to seventeen times faster. By returning the secret, its door, and supporting evidence in one result, SSA turns credential detection into an actionable finding for triage and remediation.