# Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs

arXiv: https://arxiv.org/abs/2609.00184 · HTML: https://mlsanity.com/paper/2609.00184/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 40 of its 55 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 30 of 40)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning](https://mlsanity.com/paper/2410.07163/) (2024)
- [A Comprehensive Study of Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2401.01286/) (2024)
- [Emergent and Predictable Memorization in Large Language Models](https://mlsanity.com/paper/2304.11158/) (2023)
- [StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models](https://mlsanity.com/paper/2205.11388/) (2022)
- [TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models](https://mlsanity.com/paper/2204.14211/) (2022)
- [Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora](https://mlsanity.com/paper/2110.08534/) (2021)
- [WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models](https://mlsanity.com/paper/2405.14768/) (2024)
- [Time Waits for No One! Analysis and Challenges of Temporal Misalignment](https://mlsanity.com/paper/2111.07408/) (2021)
- [On the Diversity of Synthetic Data and its Impact on Training Large Language Models](https://mlsanity.com/paper/2410.15226/) (2024)
- [Demystifying Verbatim Memorization in Large Language Models](https://mlsanity.com/paper/2407.17817/) (2024)
- [DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows](https://mlsanity.com/paper/2402.10379/) (2024)
- [Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases](https://mlsanity.com/paper/2502.19249/) (2025)
- [In-Context Editing: Learning Knowledge from Self-Induced Distributions](https://mlsanity.com/paper/2406.11194/) (2024)
- [WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs](https://mlsanity.com/paper/2503.05683/) (2025)
- [NEO-BENCH: Evaluating Robustness of Large Language Models with Neologisms](https://mlsanity.com/paper/2402.12261/) (2024)
- [Knowledge Editing in Language Models via Adapted Direct Preference Optimization](https://mlsanity.com/paper/2406.09920/) (2024)
- [Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoning](https://mlsanity.com/paper/2503.05212/) (2025)
- [Evaluating Dependencies in Fact Editing for Language Models: Specificity and Implication Awareness](https://mlsanity.com/paper/2312.01858/) (2023)
- [Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models](https://mlsanity.com/paper/2402.11900/) (2024)

## Abstract (arXiv)

Large language models (LLMs) rely on static pretraining corpora, causing their knowledge to become outdated over time. Existing approaches for evaluating knowledge edits either suffer from rapid contamination or rely on counterfactual edits that conflict with rigid existing knowledge. In this work, we propose a synthetic, simulation-driven framework for studying knowledge insertion in LLMs. We introduce sc ParallelEvents, a benchmark of fictional yet realistic future worlds that generates coherent event trajectories for controlled evaluation, avoiding contamination while preserving consistency. Building on this dataset, we develop sc Synapse, a training framework that uses model-generated data to update model parameters via mid-training and instruction tuning. This synthetic pipeline enables scalable knowledge integration without costly human-curated data. Empirically, sc Synapse outperforms existing methods by 14.23%, demonstrating that simulation-based synthetic training leads to robust and coherent knowledge insertions.