# Consistent estimation of generative model representations in the data kernel perspective space

arXiv: https://arxiv.org/abs/2409.17308 · HTML: https://mlsanity.com/paper/2409.17308/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance](https://mlsanity.com/paper/2204.08583/) (2022)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [Planning with Large Language Models for Code Generation](https://mlsanity.com/paper/2303.05510/) (2023)
- [Perspectives on Large Language Models for Relevance Judgment](https://mlsanity.com/paper/2304.09161/) (2023)
- [Deep Voice: Real-time Neural Text-to-Speech](https://mlsanity.com/paper/1702.07825/) (2017)
- [GraSPy: Graph Statistics in Python](https://mlsanity.com/paper/1904.05329/) (2019)
- [Comparing Foundation Models using Data Kernels](https://mlsanity.com/paper/2305.05126/) (2023)
- [Tracking the perspectives of interacting language models](https://mlsanity.com/paper/2406.11938/) (2024)
- [Mental State Classification Using Multi-graph Features](https://mlsanity.com/paper/2203.00516/) (2022)
- [Inducing a hierarchy for multi-class classification problems](https://mlsanity.com/paper/2102.10263/) (2021)
- [Continuous Multidimensional Scaling](https://mlsanity.com/paper/2402.04436/) (2024)

## Built on by (most cited first, 10 of 10)

- [Statistical inference on black-box generative models in the data kernel perspective space](https://mlsanity.com/paper/2410.01106/) (2024)
- [Investigating social alignment via mirroring in a system of interacting language models](https://mlsanity.com/paper/2412.06834/) (2024)
- [Detecting Perspective Shifts in Multi-agent Systems](https://mlsanity.com/paper/2512.05013/) (2025)
- [Query-efficient model evaluation using cached responses](https://mlsanity.com/paper/2605.07096/) (2026)
- [Toward a digital twin of U.S. Congress](https://mlsanity.com/paper/2505.00006/) (2025)
- [Concentration bounds on response-based vector embeddings of black-box generative models](https://mlsanity.com/paper/2511.08307/) (2025)
- [Black-box model classification under the discriminative factorization](https://mlsanity.com/paper/2605.07878/) (2026)
- [Control Charts for Multi-agent Systems](https://mlsanity.com/paper/2605.11135/) (2026)
- [Data Kernel Perspective Space Performance Guarantees for Synthetic Data from Transformer Models](https://mlsanity.com/paper/2602.05106/) (2026)
- [Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models](https://mlsanity.com/paper/2605.26409/) (2026)

## Abstract (arXiv)

Generative models, such as large language models and text-to-image diffusion models, produce relevant information when presented a query. Different models may produce different information when presented the same query. As the landscape of generative models evolves, it is important to develop techniques to study and analyze differences in model behaviour. In this paper we present novel theoretical results for embedding-based representations of generative models in the context of a set of queries. In particular, we establish sufficient conditions for the consistent estimation of the model embeddings in situations where the query set and the number of models grow.