# Supernova Event Dataset: Interpreting Large Language Models' Personality through Critical Event Analysis

arXiv: https://arxiv.org/abs/2506.12189 · HTML: https://mlsanity.com/paper/2506.12189/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 13 Jun 2025.

## Builds on (most cited first, 27 of 27)

- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack](https://mlsanity.com/paper/2406.10149/) (2024)
- [Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models](https://mlsanity.com/paper/2402.14207/) (2024)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [NoLiMa: Long-Context Evaluation Beyond Literal Matching](https://mlsanity.com/paper/2502.05167/) (2025)
- [No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding](https://mlsanity.com/paper/2503.05061/) (2025)
- [LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion](https://mlsanity.com/paper/2501.15089/) (2025)
- [Strong and weak alignment of large language models with human values](https://mlsanity.com/paper/2408.04655/) (2024)
- [Sparks of Science: Hypothesis Generation Using Structured Paper Data](https://mlsanity.com/paper/2504.12976/) (2025)
- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [Personality testing of Large Language Models: Limited temporal stability, but highlighted prosociality](https://mlsanity.com/paper/2306.04308/) (2023)
- [Automatic Event Salience Identification](https://mlsanity.com/paper/1809.00647/) (2018)
- [Salience-Aware Event Chain Modeling for Narrative Understanding](https://mlsanity.com/paper/2109.10475/) (2021)
- [Writing Style Matters: An Examination of Bias and Fairness in Information Retrieval Systems](https://mlsanity.com/paper/2411.13173/) (2024)
- [Do Language Models Understand Time?](https://mlsanity.com/paper/2412.13845/) (2024)
- [A Matter of Perspective(s): Contrasting Human and LLM Argumentation in Subjective Decision-Making on Subtle Sexism](https://mlsanity.com/paper/2502.14052/) (2025)
- [A Survey on Enhancing Causal Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2503.09326/) (2025)
- [LocalValueBench: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models](https://mlsanity.com/paper/2408.01460/) (2024)
- [AI Benchmarks and Datasets for LLM Evaluation](https://mlsanity.com/paper/2412.01020/) (2024)
- [Modeling Event Salience in Narratives via Barthes' Cardinal Functions](https://mlsanity.com/paper/2011.01785/) (2020)
- [EventRL: Enhancing Event Extraction with Outcome Supervision for Large Language Models](https://mlsanity.com/paper/2402.11430/) (2024)
- [Investigating the Impact of LLM Personality on Cognitive Bias Manifestation in Automated Decision-Making Tasks](https://mlsanity.com/paper/2502.14219/) (2025)
- [Guidelines and a Corpus for Extracting Biographical Events](https://mlsanity.com/paper/2206.03547/) (2022)
- [Document-Level Event Extraction with Definition-Driven ICL](https://mlsanity.com/paper/2408.05566/) (2024)

## Built on by (most cited first, 1 of 1)

- [Visual-TableQA: Open-Domain Benchmark for Reasoning over Table Images](https://mlsanity.com/paper/2509.07966/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly integrated into everyday applications. As their influence grows, understanding their decision making and underlying personality becomes essential. In this work, we interpret model personality using our proposed Supernova Event Dataset, a novel dataset with diverse articles spanning biographies, historical events, news, and scientific discoveries. We use this dataset to benchmark LLMs on extracting and ranking key events from text, a subjective and complex challenge that requires reasoning over long-range context and modeling causal chains. We evaluate small models like Phi-4, Orca 2, and Qwen 2.5, and large, stronger models such as Claude 3.7, Gemini 2.5, and OpenAI o3, and propose a framework where another LLM acts as a judge to infer each model's personality based on its selection and classification of events. Our analysis shows distinct personality traits: for instance, Orca 2 demonstrates emotional reasoning focusing on interpersonal dynamics, while Qwen 2.5 displays a more strategic, analytical style. When analyzing scientific discovery events, Claude Sonnet 3.7 emphasizes conceptual framing, Gemini 2.5 Pro prioritizes empirical validation, and o3 favors step-by-step causal reasoning. This analysis improves model interpretability, making them user-friendly for a wide range of diverse applications. Project Page - https://www.supernova-event.ai/