# Domain adaptation of large language models for geotechnical applications

arXiv: https://arxiv.org/abs/2507.05613 · HTML: https://mlsanity.com/paper/2507.05613/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 59 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [GeoGalactica: A Scientific Large Language Model in Geoscience](https://mlsanity.com/paper/2401.00434/) (2023)
- [Rethinking Scanning Strategies with Vision Mamba in Semantic Segmentation of Remote Sensing Imagery: An Experimental Study](https://mlsanity.com/paper/2405.08493/) (2024)
- [Can ChatGPT implement finite element models for geotechnical engineering applications?](https://mlsanity.com/paper/2501.02199/) (2025)
- [Geotechnical Parrot Tales (GPT): Harnessing Large Language Models in geotechnical engineering](https://mlsanity.com/paper/2304.02138/) (2023)
- [Future-proofing geotechnics workflows: accelerating problem-solving with large language models](https://mlsanity.com/paper/2312.12411/) (2023)
- [Selection of contributing factors for predicting landslide susceptibility using machine learning and deep learning models](https://mlsanity.com/paper/2309.06062/) (2023)
- [Enhancing Environmental Monitoring through Multispectral Imaging: The WasteMS Dataset for Semantic Segmentation of Lakeside Waste](https://mlsanity.com/paper/2407.17028/) (2024)
- [Classification of Geological Borehole Descriptions Using a Domain Adapted Large Language Model](https://mlsanity.com/paper/2407.10991/) (2024)
- [GeoSim.AI: AI assistants for numerical simulations in geomechanics](https://mlsanity.com/paper/2501.14186/) (2025)
- [Educational impacts of generative artificial intelligence on learning and performance of engineering students in China](https://mlsanity.com/paper/2505.09208/) (2025)

## Built on by (most cited first, 1 of 1)

- [Tabular foundation model for GEOAI benchmark problems BM/AirportSoilProperties/2/2025](https://mlsanity.com/paper/2509.03191/) (2025)

## Abstract (arXiv)

The rapid advancement of large language models (LLMs) is transforming opportunities in geotechnical engineering, where workflows rely on complex, text-rich data. While general-purpose LLMs demonstrate strong reasoning capabilities, their effectiveness in geotechnical applications is constrained by limited exposure to specialized terminology and domain logic. Thus, domain adaptation, tailoring general LLMs for geotechnical use, has become essential. This paper presents the first systematic review of LLM adaptation and application in geotechnical contexts. It critically examines four key adaptation strategies, including prompt engineering, retrieval augmented generation, domain-adaptive pretraining, and fine-tuning, and evaluates their comparative benefits, limitations, and implementation trends. This review synthesizes current applications spanning geological interpretation, subsurface characterization, design analysis, numerical modeling, risk assessment, and geotechnical education. Findings show that domain-adapted LLMs substantially improve reasoning accuracy, automation, and interpretability, yet remain limited by data scarcity, validation challenges, and explainability concerns. Future research directions are also suggested. This review establishes a critical foundation for developing geotechnically literate LLMs and guides researchers and practitioners in advancing the digital transformation of geotechnical engineering.