# Overcoming Copyright Barriers in Corpus Distribution Through Non-Reversible Hashing

arXiv: https://arxiv.org/abs/2604.23412 · HTML: https://mlsanity.com/paper/2604.23412/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 42 references that are in the map.
- First posted on arXiv 25 Apr 2026.

## Builds on (most cited first, 8 of 8)

- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Mind the Gap: Assessing Temporal Generalization in Neural Language Models](https://mlsanity.com/paper/2102.01951/) (2021)
- [Extracting books from production language models](https://mlsanity.com/paper/2601.02671/) (2026)
- [The Project Dialogism Novel Corpus: A Dataset for Quotation Attribution in Literary Texts](https://mlsanity.com/paper/2204.05836/) (2022)
- [Serial Speakers: a Dataset of TV Series](https://mlsanity.com/paper/2002.06923/) (2020)
- [The Elephant in the Coreference Room: Resolving Coreference in Full-Length French Fiction Works](https://mlsanity.com/paper/2510.15594/) (2025)
- [The Role of Natural Language Processing Tasks in Automatic Literary Character Network Construction](https://mlsanity.com/paper/2412.11560/) (2024)
- [CREFT: Sequential Multi-Agent LLM for Character Relation Extraction](https://mlsanity.com/paper/2505.24553/) (2025)

## Abstract (arXiv)

While annotated corpora are crucial in the field of natural language processing (NLP), those containing copyrighted material are difficult to exchange among researchers. Yet, such corpora are necessary to fully represent the diversity of data found in the wild in the context of NLP tasks. We tackle this issue by proposing a method to lawfully and publicly share the annotations of copyrighted literary texts. The corpus creator shares the annotations in clear, along with a non-reversible hashed version of the source material. The corpus user must own the source material, and apply the same hash function to their own tokens, in order to match them to the shared annotations. Crucially, our method is robust to reasonable divergences in the version of the copyrighted data owned by the user. As an illustration, we present alignment experiments on different editions of novels. Our results show that our method is able to correctly align 98.7 to 99.79% of tokens depending on the novel, provided the user version is sufficiently close to the corpus creator's version. We publicly release novelshare, a Python implementation of our method.