# Free-text Keystroke Authentication using Transformers: A Comparative Study of Architectures and Loss Functions

arXiv: https://arxiv.org/abs/2310.11640 · HTML: https://mlsanity.com/paper/2310.11640/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 18 Oct 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Decoupled Contrastive Learning](https://mlsanity.com/paper/2110.06848/) (2021)
- [TypeNet: Deep Learning Keystroke Biometrics](https://mlsanity.com/paper/2101.05570/) (2021)
- [SetMargin Loss applied to Deep Keystroke Biometrics with Circle Packing Interpretation](https://mlsanity.com/paper/2109.00938/) (2021)
- [Fast Free-text Authentication via Instance-based Keystroke Dynamics](https://mlsanity.com/paper/2006.09337/) (2020)
- [BehavePassDB: Public Database for Mobile Behavioral Biometrics and Benchmark Evaluation](https://mlsanity.com/paper/2206.02502/) (2022)
- [Free-Text Keystroke Dynamics for User Authentication](https://mlsanity.com/paper/2107.07009/) (2021)
- [Mobile Keystroke Biometrics Using Transformers](https://mlsanity.com/paper/2207.07596/) (2022)
- [Keystroke Dynamics: Concepts, Techniques, and Applications](https://mlsanity.com/paper/2303.04605/) (2023)

## Built on by (most cited first, 1 of 1)

- [Template Ageing and Longitudinal Verification in Fixed-Text Keystroke Dynamics: A Subject-Disjoint Study Across Eight Weeks](https://mlsanity.com/paper/2609.29851/) (2026)

## Abstract (arXiv)

Keystroke biometrics is a promising approach for user identification and verification, leveraging the unique patterns in individuals' typing behavior. In this paper, we propose a Transformer-based network that employs self-attention to extract informative features from keystroke sequences, surpassing the performance of traditional Recurrent Neural Networks. We explore two distinct architectures, namely bi-encoder and cross-encoder, and compare their effectiveness in keystroke authentication. Furthermore, we investigate different loss functions, including triplet, batch-all triplet, and WDCL loss, along with various distance metrics such as Euclidean, Manhattan, and cosine distances. These experiments allow us to optimize the training process and enhance the performance of our model. To evaluate our proposed model, we employ the Aalto desktop keystroke dataset. The results demonstrate that the bi-encoder architecture with batch-all triplet loss and cosine distance achieves the best performance, yielding an exceptional Equal Error Rate of 0.0186%. Furthermore, alternative algorithms for calculating similarity scores are explored to enhance accuracy. Notably, the utilization of a one-class Support Vector Machine reduces the Equal Error Rate to an impressive 0.0163%. The outcomes of this study indicate that our model surpasses the previous state-of-the-art in free-text keystroke authentication. These findings contribute to advancing the field of keystroke authentication and offer practical implications for secure user verification systems.