# FicSim: A Dataset for Multi-Faceted Semantic Similarity in Long-Form Fiction

arXiv: https://arxiv.org/abs/2510.20926 · HTML: https://mlsanity.com/paper/2510.20926/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 65 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Builds on (most cited first, 22 of 22)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [∞Bench: Extending Long Context Evaluation Beyond 100K Tokens](https://mlsanity.com/paper/2402.13718/) (2024)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://mlsanity.com/paper/2407.19669/) (2024)
- [BookSum: A Collection of Datasets for Long-form Narrative Summarization](https://mlsanity.com/paper/2105.08209/) (2021)
- [HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly](https://mlsanity.com/paper/2410.02694/) (2024)
- [Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering](https://mlsanity.com/paper/1908.08167/) (2019)
- [Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension](https://mlsanity.com/paper/2203.13947/) (2022)
- [One Thousand and One Pairs: A "novel" challenge for long-context language models](https://mlsanity.com/paper/2406.16264/) (2024)
- [DE-COP: Detecting Copyrighted Content in Language Models Training Data](https://mlsanity.com/paper/2402.09910/) (2024)
- [Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture](https://mlsanity.com/paper/2310.12109/) (2023)
- [An Annotated Dataset of Coreference in English Literature](https://mlsanity.com/paper/1912.01140/) (2019)
- [LongEmbed: Extending Embedding Models for Long Context Retrieval](https://mlsanity.com/paper/2404.12096/) (2024)
- [Promises and pitfalls of artificial intelligence for legal applications](https://mlsanity.com/paper/2402.01656/) (2024)
- [Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications](https://mlsanity.com/paper/2409.18454/) (2024)
- [Topic Modeling Genre: An Exploration of French Classical and Enlightenment Drama](https://mlsanity.com/paper/2103.13019/) (2021)
- [On Classification with Large Language Models in Cultural Analytics](https://mlsanity.com/paper/2410.12029/) (2024)
- [Computational thematics: Comparing algorithms for clustering the genres of literary fiction](https://mlsanity.com/paper/2305.11251/) (2023)
- [Beyond Text: Characterizing Domain Expert Needs in Document Research](https://mlsanity.com/paper/2504.12495/) (2025)

## Built on by (most cited first, 1 of 1)

- [Social Story Frames: Contextual Reasoning about Narrative Intent and Reception](https://mlsanity.com/paper/2512.15925/) (2025)

## Abstract (arXiv)

As language models become capable of processing increasingly long and complex texts, there has been growing interest in their application within computational literary studies. However, evaluating the usefulness of these models for such tasks remains challenging due to the cost of fine-grained annotation for long-form texts and the data contamination concerns inherent in using public-domain literature. Current embedding similarity datasets are not suitable for evaluating literary-domain tasks because of a focus on coarse-grained similarity and primarily on very short text. We assemble and release FICSIM, a dataset of long-form, recently written fiction, including scores along 12 axes of similarity informed by author-produced metadata and validated by digital humanities scholars. We evaluate a suite of embedding models on this task, demonstrating a tendency across models to focus on surface-level features over semantic categories that would be useful for computational literary studies tasks. Throughout our data-collection process, we prioritize author agency and rely on continual, informed author consent.