# Conditional Hypothesis Generation for LLM-Based Text Analysis with Researcher-Specified Covariates

arXiv: https://arxiv.org/abs/2606.03029 · HTML: https://mlsanity.com/paper/2606.03029/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 47 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Builds on (most cited first, 9 of 9)

- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Goal Driven Discovery of Distributional Differences via Language Descriptions](https://mlsanity.com/paper/2302.14233/) (2023)
- [Describing Differences between Text Distributions with Natural Language](https://mlsanity.com/paper/2201.12323/) (2022)
- [The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcripts](https://mlsanity.com/paper/2211.11772/) (2022)
- [Explaining Datasets in Words: Statistical Models with Natural Language Parameters](https://mlsanity.com/paper/2409.08466/) (2024)
- [Literature Meets Data: A Synergistic Approach to Hypothesis Generation](https://mlsanity.com/paper/2410.17309/) (2024)
- [Position: Use Sparse Autoencoders to Discover Unknowns](https://mlsanity.com/paper/2506.23845/) (2025)
- [The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Education](https://mlsanity.com/paper/2404.02444/) (2024)
- [Causal Inference on Outcomes Learned from Text](https://mlsanity.com/paper/2503.00725/) (2025)

## Abstract (arXiv)

A core goal of computational social science is to discover interpretable differences in how language varies across outcomes of interest, such as political affiliation or instructional quality. Recent LLM-based hypothesis generation methods describe such differences in natural language, but select for globally discriminative patterns without accounting for covariates that shape the data based on researchers' domain knowledge. When covariates are ignored, selected patterns can reflect confounds rather than differences of substantive interest. We introduce conditional hypothesis generation, a framework that incorporates researcher-specified covariates to steer hypothesis discovery toward differences that hold within relevant subgroups. Two challenges arise: the target subgroup may be underrepresented (stratum imbalance), and the direction of a difference may reverse across subgroups (sign reversal). We propose two econometrics-inspired methods: one introduces feature--covariate interactions to detect sign reversals, and the other applies within-stratum demeaning and inverse-frequency reweighting to equalize underrepresented strata. Synthetic experiments show each method outperforms global baselines in its targeted setting, and expert evaluation on two real-world datasets confirms that covariate-aware generation surfaces more useful hypotheses within relevant subgroups.