# Making Political Text Scaling Comparable: Infrastructure and Hyperparameter Sensitivity for 17 Algorithms

arXiv: https://arxiv.org/abs/2609.17602 · HTML: https://mlsanity.com/paper/2609.17602/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 46 references that are in the map.
- First posted on arXiv 13 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [Large Language Models Can Be Used to Estimate the Latent Positions of Politicians](https://mlsanity.com/paper/2303.12057/) (2023)
- [Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts](https://mlsanity.com/paper/2407.10747/) (2024)
- [Text-Based Ideal Points](https://mlsanity.com/paper/2005.04232/) (2020)
- [Political Text Scaling Meets Computational Semantics](https://mlsanity.com/paper/1904.06217/) (2019)
- [A Public Dataset Tracking Social Media Discourse about the 2024 U.S. Presidential Election on Twitter/X](https://mlsanity.com/paper/2411.00376/) (2024)
- [Bridging Dictionary: AI-Generated Dictionary of Partisan Language Use](https://mlsanity.com/paper/2407.09661/) (2024)
- [SpeakGer: A meta-data enriched speech corpus of German state and federal parliaments](https://mlsanity.com/paper/2410.17886/) (2024)
- [Wordkrill: Extending Wordfish into the multidimensional political space](https://mlsanity.com/paper/2506.20275/) (2025)
- [Computational Measurement of Political Positions: A Review of Text-Based Ideal Point Estimation Algorithms](https://mlsanity.com/paper/2511.13238/) (2025)

## Abstract (arXiv)

Computational text-based ideal point estimation (CT-IPE) methods are usually compared as named algorithms, yet applying them involves numerous researcher choices that configure how political text is turned into position estimates. This paper argues that CT-IPE methods are better understood as configurable measurement pipelines than as fixed estimators. Building on a large-scale comparative experiment spanning 17 CT-IPE algorithms, 5,537 experimental runs, and approximately 4.25 million left-right position estimates, I describe the shared infrastructure that makes these heterogeneous methods jointly executable and quantify how sensitive their estimates are to alternative hyperparameter choices. Variance-partitioning and SHAP-based sensitivity analyses show that, for most algorithms, hyperparameter profiles explain little residual variance through a shared shift: 13 of the 17 algorithms exhibit ICC values below .10. Where this profile-level sensitivity is present, it is concentrated in a small number of consequential researcher choices, most notably the selection of the underlying language or embedding model, the seed keyword lists that anchor the construct, and the number of topics.