# StoryScope: Investigating idiosyncrasies in AI fiction

arXiv: https://arxiv.org/abs/2604.03136 · HTML: https://mlsanity.com/paper/2604.03136/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 72 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Builds on (most cited first, 30 of 46)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [Self-Alignment with Instruction Backtranslation](https://mlsanity.com/paper/2308.06259/) (2023)
- [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://mlsanity.com/paper/2510.22954/) (2025)
- [EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models](https://mlsanity.com/paper/2312.06281/) (2023)
- [Are Large Language Models Capable of Generating Human-Level Narratives?](https://mlsanity.com/paper/2407.13248/) (2024)
- [Delving into LLM-assisted writing in biomedical publications through excess vocabulary](https://mlsanity.com/paper/2406.07016/) (2024)
- [Do LLMs write like humans? Variation in grammatical and rhetorical styles](https://mlsanity.com/paper/2410.16107/) (2024)
- [Goal-Driven Explainable Clustering via Language Descriptions](https://mlsanity.com/paper/2305.13749/) (2023)
- [On the Creativity of Large Language Models](https://mlsanity.com/paper/2304.00008/) (2023)
- [Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits](https://mlsanity.com/paper/2409.14509/) (2024)
- [Echoes in AI: Quantifying lack of plot diversity in LLM outputs](https://mlsanity.com/paper/2501.00273/) (2024)
- [Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges](https://mlsanity.com/paper/2408.08946/) (2024)
- [People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text](https://mlsanity.com/paper/2501.15654/) (2025)
- [Training-free LLM-generated Text Detection by Mining Token Probability Sequences](https://mlsanity.com/paper/2410.06072/) (2024)
- [Technical Report on the Pangram AI-Generated Text Classifier](https://mlsanity.com/paper/2402.14873/) (2024)
- [Your Large Language Models Are Leaving Fingerprints](https://mlsanity.com/paper/2405.14057/) (2024)
- [Idiosyncrasies in Large Language Models](https://mlsanity.com/paper/2502.12150/) (2025)
- [Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling](https://mlsanity.com/paper/2310.12902/) (2023)
- [EditLens: Quantifying the Extent of AI Editing in Text](https://mlsanity.com/paper/2510.03154/) (2025)
- [Imitate Before Detect: Aligning Machine Stylistic Preference for Machine-Revised Text Detection](https://mlsanity.com/paper/2412.10432/) (2024)
- [QUDsim: Quantifying Discourse Similarities in LLM-Generated Text](https://mlsanity.com/paper/2504.09373/) (2025)
- [NarraBench: A Comprehensive Framework for Narrative Benchmarking](https://mlsanity.com/paper/2510.09869/) (2025)

## Built on by (most cited first, 9 of 9)

- [Pangram 4 Technical Report](https://mlsanity.com/paper/2607.27183/) (2026)
- [AI as a Tool for Simulation-Based Experiments in Literary Studies](https://mlsanity.com/paper/2606.02293/) (2026)
- [Generative AI floods and dilutes the market for books](https://mlsanity.com/paper/2607.20349/) (2026)
- [Argument Collapse: LLMs Flatten Long-Form Public Debate](https://mlsanity.com/paper/2606.01736/) (2026)
- [Measuring, Localizing, and Ablating Alignment Signatures in LLMs](https://mlsanity.com/paper/2605.30526/) (2026)
- [POLARIS: Guiding Small Models to Write Long Stories](https://mlsanity.com/paper/2606.04095/) (2026)
- [GENIE: A Fine-Grained Measure for Novelty](https://mlsanity.com/paper/2606.12790/) (2026)
- [The House with a Million Windows: Interactive Fiction for Narrative Restorying](https://mlsanity.com/paper/2609.12537/) (2026)
- [How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text](https://mlsanity.com/paper/2609.40295/) (2026)

## Abstract (arXiv)

As AI-generated fiction becomes increasingly prevalent, questions of authorship and originality are becoming central to how written work is evaluated. While most existing work in this space focuses on identifying surface-level signatures of AI writing, we ask instead whether AI-generated stories can be distinguished from human ones without relying on stylistic signals, focusing on discourse-level narrative choices such as character agency and chronological discontinuity. We propose StoryScope, a pipeline that automatically induces a fine-grained, interpretable feature space of discourse-level narrative features across 10 dimensions. We apply StoryScope to a parallel corpus of 10,272 writing prompts, each written by a human author and five LLMs, yielding 61,608 stories, each ~5,000 words, and 304 extracted features per story. Narrative features alone achieve 93.2% macro-F1 for human vs. AI detection and 68.4% macro-F1 for six-way authorship attribution, retaining over 97% of the performance of models that include stylistic cues. A compact set of 30 core narrative features captures much of this signal: AI stories over-explain themes and favor tidy, single-track plots while human stories frame protagonist' choices as more morally ambiguous and have increased temporal complexity. Per-model fingerprint features enable six-way attribution: for example, Claude produces notably flat event escalation, GPT over-indexes on dream sequences, and Gemini defaults to external character description. We find that AI-generated stories cluster in a shared region of narrative space, while human-authored stories exhibit greater diversity. More broadly, these results suggest that differences in underlying narrative construction, not just writing style, can be used to separate human-written original works from AI-generated fiction.