# Flexible metadata harvesting for ecology using large language models

arXiv: https://arxiv.org/abs/2508.20115 · HTML: https://mlsanity.com/paper/2508.20115/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Ragas: Automated Evaluation of Retrieval Augmented Generation](https://mlsanity.com/paper/2309.15217/) (2023)
- [LightRAG: Simple and Fast Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.05779/) (2024)
- [Croissant: A Metadata Format for ML-Ready Datasets](https://mlsanity.com/paper/2403.19546/) (2024)
- [Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning](https://mlsanity.com/paper/2304.02711/) (2023)
- [Link Climate: An Interoperable Knowledge Graph Platform for Climate Data](https://mlsanity.com/paper/2210.16050/) (2022)
- [Lost or found? Discovering data needed for research](https://mlsanity.com/paper/1909.00464/) (2019)
- [Utilising a Large Language Model to Annotate Subject Metadata: A Case Study in an Australian National Research Data Catalogue](https://mlsanity.com/paper/2310.11318/) (2023)
- [Metadata-based Data Exploration with Retrieval-Augmented Generation for Large Language Models](https://mlsanity.com/paper/2410.04231/) (2024)
- [Making Metadata More FAIR Using Large Language Models](https://mlsanity.com/paper/2307.13085/) (2023)
- [Dataset Search In Biodiversity Research: Do Metadata In Data Repositories Reflect Scholarly Information Needs?](https://mlsanity.com/paper/2002.12021/) (2020)

## Built on by (most cited first, 1 of 1)

- [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025)

## Abstract (arXiv)

Large, open datasets can accelerate ecological research, particularly by enabling researchers to develop new insights by reusing datasets from multiple sources. However, to find the most suitable datasets to combine and integrate, researchers must navigate diverse ecological and environmental data provider platforms with varying metadata availability and standards. To overcome this obstacle, we have developed a large language model (LLM)-based metadata harvester that flexibly extracts metadata from any dataset's landing page, and converts these to a user-defined, unified format using existing metadata standards. We validate that our tool is able to extract both structured and unstructured metadata with equal accuracy, aided by our LLM post-processing protocol. Furthermore, we utilise LLMs to identify links between datasets, both by calculating embedding similarity and by unifying the formats of extracted metadata to enable rule-based processing. Our tool, which flexibly links the metadata of different datasets, can therefore be used for ontology creation or graph-based queries, for example, to find relevant ecological and environmental datasets in a virtual research environment.