mlsanityData to 14 Oct 2026

The Model Knows Your Project, Not You: Measuring Recognition in LLMs with NameRank

arXiv 2607.12520 ↗First posted 14 Jul 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

What a frontier model recalls about a person or tool from its own weights -- before any retrieval step -- often shapes the first description a human sees, making that parametric corpus presence a measurement problem. Citations explain about a third of whether a model recognizes a researcher; we target the residual and build NameRank, a [0,1] recognition score: each of 4,685 entities in 54 cohorts is probed with one open-ended question across 36 models, and an independent judge returns a binary verdict against a curated gold -- did the model state a specific, non-guessable fact about this exact entity? -- so hallucination, context echo, and guesses earn nothing. Synthetic-null entities hold the floor near zero, and verdicts track the entity, not the model. One thesis organizes the findings: recognition is paid to named, indexable artifacts, not to credentials or titles. Every Olympic-style credential sits below a working-researcher baseline, because no named artifact ships with the medal, yet the ranking inverts at the marquee tier, where Nobel, Turing, and Fields laureates saturate the panel. For independent creators the tool out-ranks its maker, and the credential that does propagate is a named method or awarded paper. Being one of many named contributors to a celebrated artifact, by contrast, earns almost nothing -- the authors listed on a flagship model report or system card sit near the recognition floor -- because recognition attaches to the artifact's own distinctive name, not to the roster behind it. No bibliometric predicts recognition well; top-density institutions out-recognize peers at matched citations; and on 258 news events recognition loads on peak salience, not persistence. A self-report probe shows introspection reads a corpus prior, not its own knowledge.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 35 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

11 of 35 references.

References · 11

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2019Language Models as Knowledge Bases?cited by 1,936
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  5. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  6. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  7. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  8. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  9. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  10. 2023Do Large Language Models Know What They Don't Know?cited by 243
  11. 2026Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacitycited by 17

Built on by

Not cited yet.