# Perceptual Reality Transformer: What Must an Illustration Preserve?

arXiv: https://arxiv.org/abs/2508.09852 · HTML: https://mlsanity.com/paper/2508.09852/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 13 Aug 2025.

## Builds on (most cited first, 9 of 9)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022)
- [SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality](https://mlsanity.com/paper/2306.14610/) (2023)
- [CLIP Behaves like a Bag-of-Words Model Cross-modally but not Uni-modally](https://mlsanity.com/paper/2502.03566/) (2025)
- [How and where does CLIP process negation?](https://mlsanity.com/paper/2407.10488/) (2024)
- [Mapping of Subjective Accounts into Interpreted Clusters (MOSAIC): Topic Modelling and LLM applied to Stroboscopic Phenomenology](https://mlsanity.com/paper/2502.18318/) (2025)
- [From dots to faces: Individual differences in visual imagery capacity predict the content of Ganzflicker-induced hallucinations](https://mlsanity.com/paper/2507.09011/) (2025)

## Abstract (arXiv)

How can models help people communicate unusual perceptual experiences without changing what they mean? A recognizable image is only part of the answer: accounts also express vividness, duration, uncertainty, and emotion. We introduce Perceptual Reality Transformer (PRT), an evidence-linked workflow and descriptive atlas that retain the source account alongside representations and generated illustrations. Three studies examine successive parts of this transformation. In 3,145 public Ganzflicker records, frozen neural representations predict complex content well (AUROC 0.887--0.894), but vividness, persistence, and unpleasantness are less recoverable; separate imagery/context responses improve vividness prediction relative to narrative embeddings alone. In 672 synthetic contrasts, similarity judgments depend strongly on wording, while exploratory probes recover some distinctions that cosine handles poorly. Finally, 72 matched illustrations from 24 reports compare direct prompting, evidence in prose, and structured PRT. The automated coverage metric does not demonstrate a structured-format advantage; near-ceiling scores, an apparent judge error, and an analogy-to-object scoring ambiguity limit its interpretation. Together, the studies motivate separating missing information, inaccessible meaning, and an inadequate evaluation criterion. The resulting atlas makes these choices inspectable, without treating generated images as measurements of private perception.