# From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models

arXiv: https://arxiv.org/abs/2509.09303 · HTML: https://mlsanity.com/paper/2509.09303/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 94 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Builds on (most cited first, 30 of 30)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering](https://mlsanity.com/paper/2406.12334/) (2024)
- [An Empirical Study on Information Extraction using Large Language Models](https://mlsanity.com/paper/2305.14450/) (2023)
- [Language Models for Text Classification: Is In-Context Learning Enough?](https://mlsanity.com/paper/2403.17661/) (2024)
- [GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain](https://mlsanity.com/paper/2109.02555/) (2021)
- [PatentBERT: Patent Classification with Fine-Tuning a pre-trained BERT Model](https://mlsanity.com/paper/1906.02124/) (2019)
- [PatentSBERTa: A Deep NLP based Hybrid Model for Patent Distance and Classification using Augmented SBERT](https://mlsanity.com/paper/2103.11933/) (2021)
- [Homophily and missing links in citation networks](https://mlsanity.com/paper/1511.07643/) (2015)
- [Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science](https://mlsanity.com/paper/2305.14310/) (2023)
- [Benchmarking LLMs' Judgments with No Gold Standard](https://mlsanity.com/paper/2411.07127/) (2024)
- [Uncertainty-Aware Fusion: An Ensemble Framework for Mitigating Hallucinations in Large Language Models](https://mlsanity.com/paper/2503.05757/) (2025)
- [PaECTER: Patent-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2402.19411/) (2024)
- [ReactIE: Enhancing Chemical Reaction Extraction with Weak Supervision](https://mlsanity.com/paper/2307.01448/) (2023)
- [Classifying Patents Based on their Semantic Content](https://mlsanity.com/paper/1612.08504/) (2016)
- [A Novel Patent Similarity Measurement Methodology: Semantic Distance and Technological Distance](https://mlsanity.com/paper/2303.16767/) (2023)
- [Mapping Technology Space by Normalizing Patent Networks](https://mlsanity.com/paper/1509.07285/) (2015)
- [Clinical Tagging with Joint Probabilistic Models](https://mlsanity.com/paper/1608.00686/) (2016)
- [Evaluating approaches to identifying research supporting the United Nations Sustainable Development Goals](https://mlsanity.com/paper/2209.07285/) (2022)
- [An analysis of the evolution of science-technology linkage in biomedicine](https://mlsanity.com/paper/1903.10610/) (2019)
- [Improving reference mining in patents with BERT](https://mlsanity.com/paper/2101.01039/) (2021)
- [Mapping Complex Technologies via Science-Technology Linkages; The Case of Neuroscience -- A transformer based keyword extraction approach](https://mlsanity.com/paper/2205.10153/) (2022)
- [Patent-publication pairs for the detection of knowledge transfer from research to industry: reducing ambiguities with word embeddings and references](https://mlsanity.com/paper/2412.00978/) (2024)

## Abstract (arXiv)

Classifying patents by their relevance to the UN Sustainable Development Goals (SDGs) is crucial for tracking how innovation addresses global challenges. However, the absence of a large, labeled dataset limits the use of supervised learning. Existing methods, such as keyword searches, transfer learning, and citation-based heuristics, lack scalability and generalizability. This paper frames patent-to-SDG classification as a weak supervision problem, using citations from patents to SDG-tagged scientific publications (NPL citations) as a noisy initial signal. To address its sparsity and noise, we develop a composite labeling function (LF) that uses large language models (LLMs) to extract structured concepts, namely functions, solutions, and applications, from patents and SDG papers based on a patent ontology. Cross-domain similarity scores are computed and combined using a rank-based retrieval approach. The LF is calibrated via a custom positive-only loss that aligns with known NPL-SDG links without penalizing discovery of new SDG associations. The result is a silver-standard, soft multi-label dataset mapping patents to SDGs, enabling the training of effective multi-label regression models. We validate our approach through two complementary strategies: (1) internal validation against held-out NPL-based labels, where our method outperforms several baselines including transformer-based models, and zero-shot LLM; and (2) external validation using network modularity in patent citation, co-inventor, and co-applicant graphs, where our labels reveal greater thematic, cognitive, and organizational coherence than traditional technological classifications. These results show that weak supervision and semantic alignment can enhance SDG classification at scale.