# TextMineX: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Action

arXiv: https://arxiv.org/abs/2509.15098 · HTML: https://mlsanity.com/paper/2509.15098/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 42 of its 84 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 30 of 42)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2311.09476/) (2023)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [Split and Merge: Aligning Position Biases in LLM-based Evaluators](https://mlsanity.com/paper/2310.01432/) (2023)
- [RankME: Reliable Human Ratings for Natural Language Generation](https://mlsanity.com/paper/1803.05928/) (2018)
- [Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases](https://mlsanity.com/paper/2009.11564/) (2020)
- [GenIE: Generative Information Extraction](https://mlsanity.com/paper/2112.08340/) (2021)
- [On the Limitations of Reference-Free Evaluations of Generated Text](https://mlsanity.com/paper/2210.12563/) (2022)
- [ExplainaBoard: An Explainable Leaderboard for NLP](https://mlsanity.com/paper/2104.06387/) (2021)
- [Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text](https://mlsanity.com/paper/2308.02357/) (2023)
- [Multi-Dimensional Evaluation of Text Summarization with In-Context Learning](https://mlsanity.com/paper/2306.01200/) (2023)
- [Chainpoll: A high efficacy method for LLM hallucination detection](https://mlsanity.com/paper/2310.18344/) (2023)
- [Evaluating Language Models as Synthetic Data Generators](https://mlsanity.com/paper/2412.03679/) (2024)
- [Better than Average: Paired Evaluation of NLP Systems](https://mlsanity.com/paper/2110.10746/) (2021)
- [A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice](https://mlsanity.com/paper/2404.16958/) (2024)
- [OPIEC: An Open Information Extraction Corpus](https://mlsanity.com/paper/1904.12324/) (2019)
- [Large Language Models for Biomedical Knowledge Graph Construction: Information extraction from EMR notes](https://mlsanity.com/paper/2301.12473/) (2023)
- [Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering](https://mlsanity.com/paper/2502.11491/) (2025)
- [PolyIE: A Dataset of Information Extraction from Polymer Material Scientific Literature](https://mlsanity.com/paper/2311.07715/) (2023)
- [Enabling LLM Knowledge Analysis via Extensive Materialization](https://mlsanity.com/paper/2411.04920/) (2024)
- [BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation](https://mlsanity.com/paper/2109.06850/) (2021)
- [Human-Centric Research for NLP: Towards a Definition and Guiding Questions](https://mlsanity.com/paper/2207.04447/) (2022)
- [Uncovering Limitations of Large Language Models in Information Seeking from Tables](https://mlsanity.com/paper/2406.04113/) (2024)

## Abstract (arXiv)

Humanitarian Mine Action (HMA) addresses the challenge of detecting and removing landmines from conflict regions. Much of the life-saving operational knowledge produced by HMA agencies is buried in unstructured reports, limiting the transferability of information between agencies. To address this issue, we propose TextMineX: the first dataset, evaluation framework and ontology-guided large language model (LLM) pipeline for knowledge extraction from text in the HMA domain. TextMineX structures HMA reports into (subject, relation, object)-triples, thus creating domain-specific knowledge. To ensure real-world relevance, we utilized the dataset from our collaborator Cambodian Mine Action Centre (CMAC). We further introduce a bias-aware evaluation framework that combines human-annotated triples with an LLM-as-Judge protocol to mitigate position bias in reference-free scoring. Our experiments show that ontology-aligned prompts improve extraction accuracy by up to 44.2%, reduce hallucinations by 22.5%, and enhance format adherence by 20.9% compared to baseline models. We publicly release the dataset and code.