# From Text to Parameters: Predicting Item Parameters from Embedding Regularization with Reliability and Design Ceilings

arXiv: https://arxiv.org/abs/2607.07141 · HTML: https://mlsanity.com/paper/2607.07141/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 39 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Instructions and Guide for Diagnostic Questions: The NeurIPS 2020 Education Challenge](https://mlsanity.com/paper/2007.12061/) (2020)
- [R2DE: a NLP approach to estimating IRT parameters of newly generated questions](https://mlsanity.com/paper/2001.07569/) (2020)
- [Reasoning and Sampling-Augmented MCQ Difficulty Prediction via LLMs](https://mlsanity.com/paper/2503.08551/) (2025)
- [Documents Are People and Words Are Items: A Psychometric Approach to Textual Data with Contextual Embeddings](https://mlsanity.com/paper/2509.08920/) (2025)

## Built on by (most cited first, 2 of 2)

- [Toward Embedding-Based Psychometrics: Structural Modeling of Assessment-Item Semantics With Contextual Scores](https://mlsanity.com/paper/2609.31976/) (2026)
- [A Unified Response Framework for Questions and Tutoring Dialogues](https://mlsanity.com/paper/2610.19532/) (2026)

## Abstract (arXiv)

Newly developed items must ordinarily be field tested before their psychometric properties are known, creating a cold start problem for item calibration. Predicting item parameters from features is a long standing measurement problem dating back to the Linear Logistic Test Model; modern text embeddings now automate the design matrices traditionally specified by hand. We propose an evaluation framework combining regularized regression on item text embeddings, repeated cross validated R squared reported with its resampling standard deviation, and two performance upper bounds: a reliability ceiling derived from parameter standard errors, and a design ceiling derived from simulation based power calibration. Applying this framework to a mathematics item bank (EEDI) and a medical licensure benchmark (BEA 2024), we find that item difficulty is highly predictable from text (repeated cross validated R squared = 0.53, or about 57% of its reliability ceiling), whereas discrimination and pseudo guessing appear less predictable. However, evaluating these results against our ceilings reveals that this apparent hierarchy stems from target reliability rather than text signal strength: text uniformly recovers 57 to 63% of the reliable variance across difficulty targets, whereas the 3PL pseudo guessing parameter has a reliability ceiling near zero, making it an unviable target at current precision. On BEA, embedding based regression matches leaderboard RMSE despite explaining almost no variance, highlighting the critical need for scale free metrics and explicit ceilings in benchmarking. Finally, we show that a single train and test split can inflate apparent accuracy by 0.1 to 0.15 in R squared, underscoring the necessity of repeated cross validation for calibration support applications and future benchmark construction.