# Behavioral Fingerprints for LLM Endpoint Stability and Identity

arXiv: https://arxiv.org/abs/2603.19022 · HTML: https://mlsanity.com/paper/2603.19022/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 11 references that are in the map.
- First posted on arXiv 19 Mar 2026.

## Builds on (most cited first, 5 of 5)

- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [Hypothesis testing with e-values](https://mlsanity.com/paper/2410.23614/) (2024)
- [IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary](https://mlsanity.com/paper/1910.12903/) (2019)
- [Token-Efficient Change Detection in LLM APIs](https://mlsanity.com/paper/2602.11083/) (2026)

## Built on by (most cited first, 4 of 4)

- [KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditing](https://mlsanity.com/paper/2605.29524/) (2026)
- [Measuring and Improving Behavioral Consistency in Large Language Models through Fact-Heuristic-Emotion State Enforcement](https://mlsanity.com/paper/2607.24765/) (2026)
- [Auditing Anonymous AI Models: A Four-Stage Protocol for Black-Box Identity Verification](https://mlsanity.com/paper/2608.31142/) (2026)
- [SafeLLM4SE: Statistical Evaluation and Reporting for LLM-based Software Engineering Systems](https://mlsanity.com/paper/2609.37294/) (2026)

## Abstract (arXiv)

The consistency of AI-native applications depends on the behavioral consistency of the model endpoints that power them. Traditional reliability metrics such as uptime, latency and throughput do not capture behavioral change, and an endpoint can remain "healthy" while its effective model identity changes due to updates to weights, tokenizers, quantization, inference engines, kernels, caching, routing, or hardware. We introduce Stability Monitor, a black-box stability monitoring system that periodically fingerprints an endpoint by sampling outputs from a fixed prompt set and comparing the resulting output distributions over time. Fingerprints are compared using a summed energy distance statistic across prompts, with permutation-test p-values as evidence of distribution shift aggregated sequentially to detect change events and define stability periods. In controlled validation, Stability Monitor detects changes to model family, version, inference stack, quantization, and behavioral parameters. In real-world monitoring of the same model hosted by multiple providers, we observe substantial provider-to-provider and within-provider stability differences.