mlsanityData to 14 Oct 2026

PETRA: Transforming Web Text for Petroleum-Engineering Domain Adaptation

arXiv 2606.24346 ↗First posted 23 Jun 2026cs.IR cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Petroleum-engineering search exposes a supervision gap for strong general retrievers: relevant evidence exists in public web text, but domain relevance labels are scarce. To address this gap, we propose PETRA, a large-scale Petroleum Engineering Text for Retrieval Adaptation dataset and pipeline that converts noisy public web data into a curated domain corpus and synthetic supervision for dense retrieval and reranking. PETRA contains 1.36M curated chunks, approximately 2B token equivalents, ≈859k, embedding training rows from ≈224k anchors, and roughly 400k teacher-scored reranker candidate rows. Its construction combines high-recall energy-domain curation, an energy-domain classifier with 98.4% test accuracy, chunk-grounded query generation, LLM-written hard negatives, and retrieval-mined candidate lists. PETRA improves first-stage in-domain Normalized Discounted Cumulative Gain (nDCG) from 0.703 to 0.763 through score fusion. Reranker adaptation improves the public Earth Science benchmark by 44% relative and a six-task reasoning-intensive panel by 23%. Failed training recipes show that high train-holdout accuracy on synthetic labels does not predict retrieval gains; retrieval-mined data helps only after being repackaged as teacher-scored candidate lists sampled from the inference-time candidate distribution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 55 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

37 of 55 references.

References · 37

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  8. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  9. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  10. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  12. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
25 more
  1. 2023LIMA: Less Is More for Alignmentcited by 1,100
  2. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  3. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  4. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  5. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  6. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  7. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  8. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  9. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  10. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 398
  11. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 376
  12. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  13. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
  14. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  15. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  16. 2023Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labelscited by 86
  17. 2023K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilizationcited by 69
  18. 2023UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankerscited by 37
  19. 2025Hard Negatives, Hard Lessons: Revisiting Training Data Quality for Robust Information Retrieval with LLMscited by 17
  20. 2024Greenback Bears and Fiscal Hawks: Finance is a Jungle and Text Embeddings Must Adaptcited by 12
  21. 2024INDUS: Effective and Efficient Language Models for Scientific Applicationscited by 6
  22. 2024RRADistill: Distilling LLMs' Passage Ranking Ability for Long-Tail Queries Document Re-Ranking on a Search Enginecited by 6
  23. 2025T-VEC: A Telecom-Specific Vectorization Model with Enhanced Semantic Understanding via Deep Triplet Loss Fine-Tuningcited by 3
  24. 2025Efficiency-Effectiveness Reranking FLOPs for LLM-based Rerankerscited by 3
  25. 2025Towards EnergyGPT: A Large Language Model Specialized for the Energy Sectorcited by 3

Built on by

Not cited yet.