mlsanityData to 7 Oct 2026

Whose record is this? Diagnosing and authorizing record use in personalized multimodal models

arXiv 2609.04801 ↗First posted 4 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Contextualized visual personalization can retrieve a true record yet apply it to the wrong visual subject. We formalize when a record may condition an answer as record authorization: subject presence (P), record-edge validity (E), and answer support (S) must all hold. We call violations visual memory misbinding (VMM). We construct RecordAuth-Diag, a 3,690-case matched diagnostic suite that changes one image--record edge while holding the query, question, record text, and image multiset fixed. Card removal and nonce relabeling attribute these failures to supplied records. Raw-bank failures span Qwen-, Phi-, and Gemma-family interfaces: Gemma-3-4B-IT reaches 63.69% local unauthorized use at 25.75% clean recall. CoViP remains at 26.02%, versus 22.49% for its Qwen backbone at similar clean recall. Typed pre-generation authorization reduces Qwen card exposure on RecordAuth-Diag from 43.63% to 3.06%, while positive recall changes from 86.26% to 60.90%. Full Pwedge Ewedge S validation uses 560 localized DAVIS cases: top-1 relevance and typed authorization have comparable release (28.93% and 28.39%) but 6.79% and 0.89% unsafe release, respectively. Of the 33 additional unsafe cases removed, 27 are support, 4 edge, 2 clean, and 0 boundary cases. Thus the observed increment is an Ewedge S decision dominated by support, not an edge check alone. Appearance supplies E evidence only conditional on P; authenticated subject tokens instantiate the missing presence witness as a sufficiency control. The claims concern the evaluated contracts, not natural prevalence, consent, or visual identity

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

23 of 26 references.

References · 23

  1. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  2. 2017Selective Classification for Deep Neural Networkscited by 799
  3. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  4. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  5. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  6. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  7. 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 238
  8. 2024RAP: Retrieval-Augmented Personalization for Multimodal Large Language Modelscited by 24
  9. 2025MMPB: It's Time for Multi-Modal Personalizationcited by 14
  10. 2024Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awarenesscited by 13
  11. 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 12
  12. 2023AMELI: Enhancing Multimodal Entity Linking with Fine-Grained Attributescited by 8
11 more
  1. 2024SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignmentcited by 8
  2. 2026GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agentscited by 8
  3. 2026MAP-Graph: Provenance-Aware Shared Memory for Multi-Agent Workflowscited by 7
  4. 2026When Memory Becomes Authority: Benchmarking Authority Collapse at the Memory Consolidation Boundarycited by 6
  5. 2026MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generationcited by 3
  6. 2026Memory Provenance Laundering in LLM Agents: A Non-Amplification Firewall for Persistent Memorycited by 3
  7. 2026OmniHalluc-L: Counterfactual Benchmarking and Modality-Perturbation Reliability Calibration for Long-Form Omni Hallucinationcited by 2
  8. 2025Unveiling the Attribute Misbinding Threat in Identity-Preserving Modelscited by 1
  9. 2026SIEVES: Selective Prediction Generalizes through Visual Evidence Scoringcited by 1
  10. 2026CAGE: Certified Authorization under Typed-Return Uncertainty for Tool-Using Agentscited by 1
  11. 2026Diagnosing Dense Same-Class Attribute Misbinding in Large Vision-Language Modelscited by 1

Built on by

Not cited yet.