# ValiText -- a unified validation framework for computational text-based measures of social constructs

arXiv: https://arxiv.org/abs/2307.02863 · HTML: https://mlsanity.com/paper/2307.02863/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2023.

## Built on by (most cited first, 2 of 2)

- [Correct codes for the wrong reasons? validating LLMs as measurement instruments for theoretical constructs](https://mlsanity.com/paper/2606.28574/) (2026)
- [The Newsworthiness of Brazilian Distress: A Peak Analysis on Time Series of International Media Attention to Disasters in Brazil](https://mlsanity.com/paper/2605.04552/) (2026)

## Abstract (arXiv)

Guidance on how to validate computational text-based measures of social constructs is fragmented. While researchers generally acknowledge the importance of validating text-based measures, they often lack a shared vocabulary and a unified framework to do so. This paper introduces ValiText, a new validation framework designed to assist scholars in validly measuring social constructs in textual data. The framework is built on a conceptual foundation of validity in the social sciences, strengthened by an empirical review of validation practices in the social sciences and consultations with experts. Ultimately, ValiText prescribes researchers to demonstrate three types of validation evidence: substantive evidence (outlining the theoretical underpinning of the measure), structural evidence (examining the properties of the text model and its output) and external evidence (testing for how the measure relates to independent information). The framework is further supplemented by a checklist of validation steps, offering practical guidance in the form of documentation sheets that guide researchers in the validation process.