mlsanityData to 14 Oct 2026

Model Correlation Detection via Random Selection Probing

arXiv 2509.24171 ↗First posted 29 Sep 2025cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

The growing prevalence of large language models (LLMs) and vision-language models (VLMs) has heightened the need for reliable techniques to determine whether a model has been fine-tuned from or is even identical to another. Existing similarity-based methods often require access to model parameters or produce heuristic scores without principled thresholds, limiting their applicability. We introduce Random Selection Probing (RSP), a hypothesis-testing framework that formulates model correlation detection as a statistical test. RSP optimizes textual or visual prefixes on a reference model for a random selection task and evaluates their transferability to a target model, producing rigorous p-values that quantify evidence of correlation. To mitigate false positives, RSP incorporates an unrelated baseline model to filter out generic, transferable features. We evaluate RSP across both LLMs and VLMs under diverse access conditions for reference models and test models. Experiments on fine-tuned and open-source models show that RSP consistently yields small p-values for related models while maintaining high p-values for unrelated ones. Extensive ablation studies further demonstrate the robustness of RSP. These results establish RSP as the first principled and general statistical framework for model correlation detection, enabling transparent and interpretable decisions in modern machine learning ecosystems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 34 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

23 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models25 Jun

Other references · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  6. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  7. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  9. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 666
  10. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  11. 2015Visual7W: Grounded Question Answering in Imagescited by 318
  12. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
10 more
  1. 2021Revisiting Model Stitching to Compare Neural Representationscited by 209
  2. 2018Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priorscited by 146
  3. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 145
  4. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 131
  5. 2025Towards Understanding Camera Motions in Any Videocited by 64
  6. 2024Hey, That's My Model! Introducing Chain & Hash, An LLM Fingerprinting Techniquecited by 37
  7. 2024REEF: Representation Encoding Fingerprints for Large Language Modelscited by 36
  8. 2021ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detectioncited by 33
  9. 2024Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Koreancited by 14
  10. 2024Building a Large Japanese Web Corpus for Large Language Modelscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Modelscited by 2
  2. 2025TokenPure: Watermark Removal through Tokenized Appearance and Structural Guidancecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition
  2. 2026Detecting and Controlling Sycophancy with Cascading Linear Features
  3. 2026ZPD Detector: Data Selection via Capability-Difficulty Alignment for Large Language Models
  4. 2026To Adapt or Not to Adapt? Selective Adaptation for Vision-Language Models
  5. 2025Language-Guided Invariance Probing of Vision-Language Models
  6. 2026Model Specific Task Similarity for Vision Language Model Selection via Layer Conductance
  7. 2025Know When to Abstain: Optimal Selective Classification with Likelihood Ratios
  8. 2024RaVL: Discovering and Mitigating Spurious Correlations in Fine-Tuned Vision-Language Models