# Embedded Conditional Independence Tests for Large Language Model Generated Text with an Application to German Parliament Speeches

arXiv: https://arxiv.org/abs/2609.00946 · HTML: https://mlsanity.com/paper/2609.00946/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 19 of 19)

- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Matryoshka Representation Learning](https://mlsanity.com/paper/2205.13147/) (2022)
- [Intrinsic Bias Metrics Do Not Correlate with Application Bias](https://mlsanity.com/paper/2012.15859/) (2020)
- [A Survey on Fairness in Large Language Models](https://mlsanity.com/paper/2308.10149/) (2023)
- [NV-Retriever: Improving text embedding models with effective hard-negative mining](https://mlsanity.com/paper/2407.15831/) (2024)
- [Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discovery](https://mlsanity.com/paper/1702.03877/) (2017)
- [LLM-based NLG Evaluation: Current Status and Challenges](https://mlsanity.com/paper/2402.01383/) (2024)
- [Using simulation studies to evaluate statistical methods](https://mlsanity.com/paper/1712.03198/) (2017)
- [BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation](https://mlsanity.com/paper/2210.07626/) (2022)
- [A general framework for inference on algorithm-agnostic variable importance](https://mlsanity.com/paper/2004.03683/) (2020)
- [The Impossibility of Fair LLMs](https://mlsanity.com/paper/2406.03198/) (2024)
- [The Projected Covariance Measure for assumption-lean variable significance testing](https://mlsanity.com/paper/2211.02039/) (2022)
- [Significance tests of feature relevance for a black-box learner](https://mlsanity.com/paper/2103.04985/) (2021)
- [New Encoders for German Trained from Scratch: Comparing ModernGBERT with Converted LLM2Vec Models](https://mlsanity.com/paper/2505.13136/) (2025)
- [Deep Nonparametric Conditional Independence Tests for Images](https://mlsanity.com/paper/2411.06140/) (2024)
- [Test and Measure for Partial Mean Dependence Based on Machine Learning Methods](https://mlsanity.com/paper/2212.12874/) (2022)
- [On the Robustness of Multilingual Text Embedding Rankings Across Learning Tasks, Languages, and Benchmark Datasets](https://mlsanity.com/paper/2605.31142/) (2026)

## Built on by (most cited first, 1 of 1)

- [Embedding-Bias in Conditional Independence Testing](https://mlsanity.com/paper/2610.11584/) (2026)

## Abstract (arXiv)

Conditional independence tests (CITs) test for conditional dependence between two random objects X and Y given a third random object Z. Existing CITs have limited applicability to high-dimensional data, especially multimodal data like text. However, we show that such tests are of interest for large language model (LLM) outputs, where we test whether an output X generated from a source text Z carries information about an attribute Y beyond Z itself. For this purpose, we propose embedded CITs (eCITs), which embed X and Z and apply an existing CIT to the resulting representations and to Y. We show that, provided the embedding of Z is sufficient, i.e. retains the information Z carries about either Y or the representation of X, the null hypothesis transfers from X and Z to their representations, so that a CIT valid for the embedded hypothesis is valid for the original one. We further give conditions for equivalence of the two hypotheses, and show that sufficiency weakens to mean sufficiency when the embedded test targets conditional mean independence. We propose a semi-synthetic simulation design to assess type I error (T1E) control and power of the eCITs for given embedding maps on a specific dataset and task, and use it to evaluate them on our application. Applying the eCITs to German Parliament speeches, we find for all combinations of embedding maps considered that the summaries of two LLMs contain information about the speaker's faction and gender beyond the speech they were generated from.