# From Rules to Neural Graphs: Scalable Structured Prediction for Patent Prior Art Search

arXiv: https://arxiv.org/abs/2610.01553 · HTML: https://mlsanity.com/paper/2610.01553/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Builds on (most cited first, 15 of 15)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Jasper and Stella: distillation of SOTA embedding models](https://mlsanity.com/paper/2412.19048/) (2024)
- [LongEmbed: Extending Embedding Models for Long Context Retrieval](https://mlsanity.com/paper/2404.12096/) (2024)
- [PaECTER: Patent-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2402.19411/) (2024)
- [PatenTEB: A Comprehensive Benchmark and Model Family for Patent Text Embedding](https://mlsanity.com/paper/2510.22264/) (2025)
- [BERT based patent novelty search by training claims to their own description](https://mlsanity.com/paper/2103.01126/) (2021)
- [Patent Search Using Triplet Networks Based Fine-Tuned SciBERT](https://mlsanity.com/paper/2207.11497/) (2022)
- [Efficient Patent Searching Using Graph Transformers](https://mlsanity.com/paper/2508.10496/) (2025)

## Abstract (arXiv)

Patent search requires processing documents routinely exceeding tens of thousands of tokens. Most neural retrieval approaches operate on truncated inputs, limiting their effectiveness. Graph-based retrieval addresses this by representing each patent as a structured invention graph, but constructing these graphs relies on brittle rule-based parsers. We present the neural parser, which adapts biaffine attention from dependency parsing to predict invention graphs directly from patent text. Our local biaffine attention restricts pairwise scoring to a sliding window, reducing complexity from O(n2) to O(n · w). Since local and global scoring share the same weights, the model trains on short sequences and deploys on documents exceeding 40,000 tokens without retraining. Distilled from 1 million rule-parsed documents, it surpasses its teacher at 3× lower inference cost: neural graphs improve citation recall by 0.5% on short queries and 1.1% on full documents in a downstream Graph Transformer retrieval system.