mlsanityData to 7 Oct 2026

LLMmap: Fingerprinting For Large Language Models

arXiv 2407.15847 ↗First posted 22 Jul 2024cs.CR cs.AIcited by 56

Not on the map. Why?

Abstract (arXiv)

We introduce LLMmap, a first-generation fingerprinting technique targeted at LLM-integrated applications. LLMmap employs an active fingerprinting approach, sending carefully crafted queries to the application and analyzing the responses to identify the specific LLM version in use. Our query selection is informed by domain expertise on how LLMs generate uniquely identifiable responses to thematically varied prompts. With as few as 8 interactions, LLMmap can accurately identify 42 different LLM versions with over 95% accuracy. More importantly, LLMmap is designed to be robust across different application layers, allowing it to identify LLM versions--whether open-source or proprietary--from various vendors, operating under various unknown system prompts, stochastic sampling hyperparameters, and even complex generation frameworks such as RAG or Chain-of-Thought. We discuss potential mitigations and demonstrate that, against resourceful adversaries, effective countermeasures may be challenging or even unrealizable.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

32 of 49 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  8. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  9. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  10. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  11. 2023Large Language Modelscited by 774
  12. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
20 more
  1. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  2. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  3. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  4. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  5. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  6. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
  7. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  8. 2023Are aligned neural networks adversarially aligned?cited by 305
  9. 2024Stealing Part of a Production Language Modelcited by 152
  10. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
  11. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  12. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 78
  13. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  14. 2024Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Modelscited by 45
  15. 2024Hey, That's My Model! Introducing Chain & Hash, An LLM Fingerprinting Techniquecited by 34
  16. 2022Geographic and Geopolitical Biases of Language Modelscited by 29
  17. 2024Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detectioncited by 27
  18. 2024A Fingerprint for Large Language Modelscited by 26
  19. 2024Buffer Overflow in Mixture of Expertscited by 10
  20. 2017OS Fingerprinting: New Techniques and a Study of Information Gain and Obfuscationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 18; 2026: 34.

56 papers.

Sort:

By area:

  1. 2024REEF: Representation Encoding Fingerprints for Large Language Modelscited by 34
  2. 2025CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Modelscited by 17
  3. 2025Auditing Black-Box LLM APIs with a Rank-Based Uniformity Testcited by 16
  4. 2025Model Provenance Testing for Large Language Modelscited by 15
  5. 2025SoK: Large Language Model Copyright Auditing via Fingerprintingcited by 14
  6. 2025SeedPrints: Fingerprints Can Even Tell Which Seed Your Large Language Model Was Trained Fromcited by 9
  7. 2025One Pic is All it Takes: Poisoning Visual Document Retrieval Augmented Generation with a Single Imagecited by 8
  8. 2025Fingerprinting LLMs via Prompt Injectioncited by 8
  9. 2026Real Money, Fake Models: Deceptive Model Claims in Shadow APIscited by 8
  10. 2025LLM Fingerprinting via Semantically Conditioned Watermarkscited by 7
  11. 2025Are Robust LLM Fingerprints Adversarially Robust?cited by 7
  12. 2024Hardware and Software Platform Inferencecited by 6
38 more
  1. 2025When AI Meets the Web: Prompt Injection Risks in Third-Party AI Chatbot Pluginscited by 6
  2. 2025Auditing Data Membership in Reinforcement Learning With Verifiable Rewardscited by 6
  3. 2026One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributionscited by 6
  4. 2025Network-Level Prompt and Trait Leakage in Local Research Agentscited by 5
  5. 2024One Prompt to Verify Your Models: Black-Box Text-to-Image Models Verification via Non-Transferable Adversarial Attackscited by 4
  6. 2025FDLLM: A Dedicated Detector for Black-Box LLMs Fingerprintingcited by 4
  7. 2025Invisible Traces: Using Hybrid Fingerprinting to identify underlying LLMs in GenAI Appscited by 4
  8. 2025Who Owns the Output? Bridging Law and Technology in LLMs Attributioncited by 4
  9. 2026Token-Efficient Change Detection in LLM APIscited by 4
  10. 2025AWM: Accurate Weight-Matrix Fingerprint for Large Language Modelscited by 3
  11. 2026Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMscited by 3
  12. 2026KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditingcited by 3
  13. 2026Cert-LAS: Toward Certified Model Ownership Verification for Text-to-Image Diffusion Models via Layer-Adaptive Smoothingcited by 3
  14. 2026Echoes within the Reasoning: Stealthy and Effective Watermarking via Chain of Thoughtcited by 2
  15. 2026Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systemscited by 2
  16. 2024Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecturecited by 1
  17. 2025Attacks and Defenses Against LLM Fingerprintingcited by 1
  18. 2026Attesting Model Lineage by Consisted Knowledge Evolution with Fine-Tuning Trajectorycited by 1
  19. 2026Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attackscited by 1
  20. 2026Robust LLM Watermarking with Minimal Semantic Distortion for IP Protectioncited by 1
  21. 2026Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Contentcited by 1
  22. 2026Fingerprinting Inference Systems of Large Language Modelscited by 1
  23. 2026Behind the Refusal: Determining Guardrail Activation via Behavioral Monitoringcited by 1
  24. 2026Blockchain Empowered Trustworthy Agent Networks: Foundations, Taxonomy, and Future Directionscited by 1
  25. 2026Scaling Model-Generated Distillation Data Can Make Latent Teacher Traits More Recoverablecited by 1
  26. 2026A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Servicescited by 1
  27. 2025Text-to-Image Models Leave Identifiable Signatures: Implications for Leaderboard Securitycited by 0
  28. 2026Identifying Models Behind Text-to-Image Leaderboardscited by 0
  29. 2026Provable Model Provenance Set for Large Language Modelscited by 0
  30. 2026Verify Claimed Text-to-Image Models via Boundary-Aware Prompt Optimizationcited by 0
  31. 2026Committed SAE-Feature Traces for Audited-Session Substitution Detection in Hosted LLMscited by 0
  32. 2026Who Owns This Agent? Tracing AI Agents Back to Their Ownerscited by 0
  33. 2026Black-Box Inference of LLM Architectural Properties with Restrictive API Accesscited by 0
  34. 2026Decision-Level Hijacking: Injecting Cognitive Bias into Large Language Models via Bit-Flip Attackscited by 0
  35. 2026Stemma: Induced Decision Regions Reveal LLM Provenancecited by 0
  36. 2026SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channelscited by 0
  37. 2026Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Spacecited by 0
  38. 2026Ventor-QTest: Threat-Model-Driven Verification of Vendor-Hosted LLM APIscited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025CoTSRF: Utilize Chain of Thought as Stealthy and Robust Fingerprint of Large Language Models
  2. 2025Behavioral Fingerprinting of Large Language Models
  3. 2023HuRef: HUman-REadable Fingerprint for Large Language Models
  4. 2026AttnDiff: Attention-based Differential Fingerprinting for Large Language Models
  5. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
  6. 2026A Behavioral Fingerprint for Large Language Models: Provenance Tracking via Refusal Vectors
  7. 2025RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation
  8. 2025Intrinsic Fingerprint of LLMs: Continue Training is NOT All You Need to Steal A Model!