# MAYA: Addressing Inconsistencies in Generative Password Guessing through a Unified Benchmark

arXiv: https://arxiv.org/abs/2504.16651 · HTML: https://mlsanity.com/paper/2504.16651/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 131 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [The Shape of Data: Intrinsic Distance for Data Distributions](https://mlsanity.com/paper/1905.11141/) (2019)
- [Towards GAN Benchmarks Which Require Generalization](https://mlsanity.com/paper/2001.03653/) (2020)
- [Manifold Topology Divergence: a Framework for Comparing Data Manifolds](https://mlsanity.com/paper/2106.04024/) (2021)
- [PassGAN: A Deep Learning Approach for Password Guessing](https://mlsanity.com/paper/1709.00440/) (2017)
- [Complexity Matters: Rethinking the Latent Space for Generative Modeling](https://mlsanity.com/paper/2307.08283/) (2023)
- [Improving Password Guessing via Representation Learning](https://mlsanity.com/paper/1910.04232/) (2019)
- [PassFlow: Guessing Passwords with Generative Flows](https://mlsanity.com/paper/2105.06165/) (2021)
- [PassGPT: Password Modeling and (Guided) Generation with Large Language Models](https://mlsanity.com/paper/2306.01545/) (2023)
- [Targeted Honeyword Generation with Language Models](https://mlsanity.com/paper/2208.06946/) (2022)
- [KeyDetect --Detection of anomalies and user based on Keystroke Dynamics](https://mlsanity.com/paper/2304.03958/) (2023)
- [PagPassGPT: Pattern Guided Password Guessing via Generative Pretrained Transformer](https://mlsanity.com/paper/2404.04886/) (2024)
- [When AI Defeats Password Deception! A Deep Learning Framework to Distinguish Passwords and Honeywords](https://mlsanity.com/paper/2407.16964/) (2024)

## Abstract (arXiv)

Recent advances in generative models have led to their application in password guessing, with the aim of replicating the complexity, structure, and patterns of human-created passwords. Despite their potential, inconsistencies and inadequate evaluation methodologies in prior research have hindered meaningful comparisons and a comprehensive, unbiased understanding of their capabilities. This paper introduces MAYA, a unified, customizable, plug-and-play benchmarking framework designed to facilitate the systematic characterization and benchmarking of generative password-guessing models in the context of trawling attacks. Using MAYA, we conduct a comprehensive assessment of six state-of-the-art approaches, which we re-implemented and adapted to ensure standardization. Our evaluation spans eight real-world password datasets and covers an exhaustive set of advanced testing scenarios, totaling over 15,000 compute hours. Our findings indicate that these models effectively capture different aspects of human password distribution and exhibit strong generalization capabilities. However, their effectiveness varies significantly with long and complex passwords. Through our evaluation, sequential models consistently outperform other generative architectures and traditional password-guessing tools, demonstrating unique capabilities in generating accurate and complex guesses. Moreover, the diverse password distributions learned by the models enable a multi-model attack that outperforms the best individual model. By releasing MAYA, we aim to foster further research, providing the community with a new tool to consistently and reliably benchmark generative password-guessing models. Our framework is publicly available at https://github.com/williamcorrias/MAYA-Password-Benchmarking.