mlsanityData to 14 Oct 2026

Statistical inference on black-box generative models in the data kernel perspective space

arXiv 2410.01106 ↗First posted 1 Oct 2024cs.LG stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Generative models are capable of producing human-expert level content across a variety of topics and domains. As the impact of generative models grows, it is necessary to develop statistical methods to understand collections of available models. These methods are particularly important in settings where the user may not have access to information related to a model's pre-training data, weights, or other relevant model-level covariates. In this paper we extend recent results on representations of black-box generative models to model-level statistical inference tasks. We demonstrate that the model-level representations are effective for multiple inference tasks.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 1 Oct 2024.

Builds on

37 of 52 references.

References · 37

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  9. 2023Mistral 7Bcited by 3,164
  10. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  11. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  12. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
25 more
  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  3. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  4. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  5. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  6. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  7. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
  8. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 265
  9. 2023AI Alignment: A Comprehensive Surveycited by 263
  10. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 237
  11. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 221
  12. 2024Observational Scaling Laws and the Predictability of Language Model Performancecited by 130
  13. 2023Foundation Models and Fair Usecited by 116
  14. 2023Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategiescited by 85
  15. 2023Efficient Benchmarking of Language Modelscited by 71
  16. 2023Unveiling the Implicit Toxicity in Large Language Modelscited by 51
  17. 2024MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answeringcited by 26
  18. 2019GraSPy: Graph Statistics in Pythoncited by 15
  19. 2023Comparing Foundation Models using Data Kernelscited by 14
  20. 2024Tracking the perspectives of interacting language modelscited by 14
  21. 2024Consistent estimation of generative model representations in the data kernel perspective spacecited by 10
  22. 2023A Statistical Turing Test for Generative Modelscited by 8
  23. 2022Mental State Classification Using Multi-graph Featurescited by 7
  24. 2020A partition-based similarity for classification distributionscited by 6
  25. 2021Inducing a hierarchy for multi-class classification problemscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025Detecting Perspective Shifts in Multi-agent Systemscited by 4
  2. 2025Gaussian mixture models as a proxy for interacting language modelscited by 2
  3. 2026Query-efficient model evaluation using cached responsescited by 2
  4. 2025Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbationscited by 1
  5. 2026Black-box model classification under the discriminative factorizationcited by 1
  6. 2026Control Charts for Multi-agent Systemscited by 1
  7. 2026Persistent Convolution: A Topological Framework for AI Alignment Testing and Semantic Space Characterizationcited by 1
  8. 2026Data Kernel Perspective Space Performance Guarantees for Synthetic Data from Transformer Modelscited by 0
  9. 2026Jailbreak susceptibility prediction and mitigation via the behavioral geometry of modelscited by 0