# LLMography: Transforming Human-AI Conversations into Traceability, Oversight, and Auditability Indicators

arXiv: https://arxiv.org/abs/2606.29437 · HTML: https://mlsanity.com/paper/2606.29437/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 28 Jun 2026.

## Builds on (most cited first, 5 of 5)

- [Generative AI in Higher Education: Seeing ChatGPT Through Universities' Policies, Resources, and Guidelines](https://mlsanity.com/paper/2312.05235/) (2023)
- [The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI supported assessment- A Preprint](https://mlsanity.com/paper/2403.14692/) (2024)
- [Higher education assessment practice in the era of generative AI tools](https://mlsanity.com/paper/2404.01036/) (2024)
- [Contra generative AI detection in higher education assessments](https://mlsanity.com/paper/2312.05241/) (2023)
- [Shaping Integrity: Why Generative Artificial Intelligence Does Not Have to Undermine Education](https://mlsanity.com/paper/2407.19088/) (2024)

## Abstract (arXiv)

The growing use of Large Language Models (LLMs) in education, software engineering, academic writing, and technical documentation raises a key question: how can we evaluate not only AI-assisted outputs, but also the interaction process that produced them? Current debates often focus on detecting whether a final artifact was generated by AI, while overlooking the conversation history that reveals human direction, AI contribution, corrections, validation, and traceability. This paper introduces LLMography, a framework for transforming Human-AI conversations into measurable indicators of provenance, human contribution, AI dependency, reproducibility, and auditability. By analogy with bibliography and webography, LLMography documents the dynamic trajectory of interaction between a human and a Large Language Model as a structured trace of Human-AI co-production. We present a prototype that analyzes Human-AI conversation traces and generates KPI reports including Prompt Quality Score, Human Direction Score, AI Dependency Level, Auditability Score, Final Output Traceability, Privacy Risk Level, and a recommended LLMography label. A preliminary exploratory evaluation was conducted on 19 anonymized audit reports from engineering students. Most interactions were classified as Human-AI co-produced, with average scores of 86.8/100 for Human Direction, 81.9/100 for Prompt Quality, 72.8/100 for Auditability, and 77.1/100 for Final Output Traceability. The paper also applies LLMography to its own writing process, classified as human-originated, human-directed, AI-assisted co-production. The findings suggest that AI transparency should move beyond output detection toward documenting the history of interaction.