# Patent Representation Learning via Self-supervision

arXiv: https://arxiv.org/abs/2511.10657 · HTML: https://mlsanity.com/paper/2511.10657/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 10 references that are in the map.
- First posted on arXiv 3 Nov 2025.

## Builds on (most cited first, 3 of 3)

- [PatentSBERTa: A Deep NLP based Hybrid Model for Patent Distance and Classification using Augmented SBERT](https://mlsanity.com/paper/2103.11933/) (2021)
- [Length-Induced Embedding Collapse in PLM-based Models](https://mlsanity.com/paper/2410.24200/) (2024)
- [Efficient Document Embeddings via Self-Contrastive Bregman Divergence Learning](https://mlsanity.com/paper/2305.16031/) (2023)

## Built on by (most cited first, 2 of 2)

- [Heterogeneous Dependency Graph-Guided Attentionfor Patent Representation Learning](https://mlsanity.com/paper/2605.10073/) (2026)
- [Asymmetric Within-Document Predictive Learning for Scientific Document Representation](https://mlsanity.com/paper/2608.28625/) (2026)

## Abstract (arXiv)

We study self-supervised patent representation learning with contrastive objectives. A standard baseline constructs positives by encoding the same text twice under independent dropout masks, but applying this recipe to long, structured patent documents requires careful calibration. We show that dropout-only training can be substantially strengthened by tuning temperature and dropout rate, yet its best configuration is evaluation-dependent and does not transfer uniformly from title--abstract retrieval to claim-to-disclosure retrieval. We propose mixed dropout--section positives, a patent-specific view construction strategy in which each positive pair links the title--abstract view of a patent either to a dropout re-encoding of itself or to another section of the same patent, such as claims, summary, background, drawings, or description. This uses patent-internal structure as a training-time signal without IPC labels, citations, or relevance annotations. We evaluate on graded EPO search-report retrieval, DAPFAM, a recently proposed family-level patent retrieval benchmark, and IPC subclass classification. Section-based positives improve over calibrated dropout-only and generic title--abstract augmentation baselines, are competitive with citation-informed patent encoders and a general-purpose embedding model, and perform strongly on the out-of-domain split of DAPFAM. Additional cross-section alignment diagnostics show that section-pair training improves compatibility among abstracts, claims, and descriptions of the same invention. These results indicate that patent sections provide effective self-supervised positive views for learning dense patent representations.