mlsanityData to 7 Oct 2026

The Ultimate Tutorial for AI-driven Scale Development in Generative Psychometrics: Releasing AIGENIE from its Bottle

arXiv 2603.28643 ↗First posted 30 Mar 2026cs.AI cs.CL cs.HCcited by 1

Not on the map. Why?

Abstract (arXiv)

Psychological scale development has traditionally required extensive expert involvement, iterative revision, and large-scale pilot testing before psychometric evaluation can begin. The textttAIGENIE R package implements the AI-GENIE framework (Automatic Item Generation and Validation with Network-Integrated Evaluation), which integrates large language model (LLM) text generation with network psychometric methods to automate the early stages of this process. The package generates candidate item pools using LLMs, transforms them into high-dimensional embeddings, and applies a multi-step reduction pipeline --- Exploratory Graph Analysis (EGA), Unique Variable Analysis (UVA), and bootstrap EGA --- to produce structurally validated item pools entirely textitin silico. This tutorial introduces the package across eight parts: installation and setup, text generation, embeddings, item generation, the full AI-GENIE pipeline, the GENIE pipeline for researcher-supplied items, advanced prompt engineering, and fully local operation. Two running examples illustrate the package's use: the Big Five personality model (a well-established construct) and AI Anxiety (an emerging construct). The package supports multiple LLM providers (OpenAI, Anthropic, Groq, HuggingFace, and local models), offers a fully offline mode with no external API calls, and provides the textttGENIE() function for researchers who wish to apply the psychometric reduction pipeline to existing item pools regardless of their origin. The textttAIGENIE package is freely available on CRAN at urlhttps://CRAN.R-project.org/package=AIGENIE.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 87 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

19 of 87 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Let's Verify Step by Stepcited by 3,598
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 229
  6. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
  7. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 168
  8. 2024Is Temperature the Creativity Parameter of Large Language Models?cited by 92
  9. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  10. 2023When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Modelscited by 50
  11. 2023Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Modelscited by 37
  12. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
7 more
  1. 2023Improved prompting and process for writing user personas with LLMs, using qualitative interviews: Capturing behaviour and personality traits of userscited by 9
  2. 2026Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Bookscited by 9
  3. 2026Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISMcited by 9
  4. 2022Evaluating the Construct Validity of Text Embeddings with Application to Survey Questionscited by 7
  5. 2025The Paradox of Stochasticity: Limited Creativity and Computational Decoupling in Temperature-Varied LLM Outputs of Structured Fictional Datacited by 6
  6. 2025The Trust in AI-Generated Health Advice (TAIGHA) Scale and Short Version (TAIGHA-S): Development and Validation Studycited by 1
  7. 2026Prompt Engineering for Scale Development in Generative Psychometricscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.