Do BERT Embeddings Encode Narrative Dimensions? A Token-Level Probing Analysis of Time, Space, Causality, and Character in Fiction
Not on the map. Why?
Abstract (arXiv)
Narrative understanding requires multidimensional semantic structures. This study investigates whether BERT embeddings encode dimensions of fictional narrative semantics -- time, space, causality, and character. Using an LLM to accelerate annotation, we construct a token-level dataset labeled with these four narrative categories plus "others." A linear probe on BERT embeddings (94% accuracy) significantly outperforms a control probe on variance-matched random embeddings (47%), confirming that BERT encodes meaningful narrative information. With balanced class weighting, the probe achieves a macro-average recall of 0.83, with moderate success on rare categories such as causality (recall = 0.75) and space (recall = 0.66). However, confusion matrix analysis reveals "Boundary Leakage," where rare dimensions are systematically misclassified as "others." Clustering analysis shows that unsupervised clustering aligns near-randomly with predefined categories (ARI = 0.081), suggesting that narrative dimensions are encoded but not as discretely separable clusters. Future work includes a POS-only baseline to disentangle syntactic patterns from narrative encoding, expanded datasets, and layer-wise probing.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 8 of its 17 references that are in the map. First posted on arXiv 12 Apr 2026.
Builds on
8 of 17 references.
References · 8
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,870
- 2018Deep contextualized word representationscited by 2,286
- 2025OpenAI GPT-5 System Cardcited by 946
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 850
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 774
- 2019Designing and Interpreting Probes with Control Taskscited by 620
- 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 437
- 2024Best Practices for Text Annotation with Large Language Modelscited by 35
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMs
- 2026Narrative World Model: Narratology-Grounded Writer Memory for Long-Form Fiction
- 2025Unveiling Intrinsic Dimension of Texts: from Academic Abstract to Creative Story
- 2024Show, Don't Tell: Uncovering Implicit Character Portrayal using LLMs
- 2023M-SENSE: Modeling Narrative Structure in Short Personal Narratives Using Protagonist's Mental Representations
- 2025EvolvTrip: Enhancing Literary Character Understanding with Temporal Theory-of-Mind Graphs
- 2023StoryGPT-V: Large Language Models as Consistent Story Visualizers
- 2025Hierarchical Knowledge Graphs for Story Understanding in Visual Narratives