# Do Sentence Transformers Learn Quasi-Geospatial Concepts from General Text?

arXiv: https://arxiv.org/abs/2404.04169 · HTML: https://mlsanity.com/paper/2404.04169/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 5 Apr 2024.

## Builds on (most cited first, 5 of 5)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Arithmetic-Based Pretraining -- Improving Numeracy of Pretrained Language Models](https://mlsanity.com/paper/2205.06733/) (2022)

## Built on by (most cited first, 3 of 3)

- [Geospatial Mechanistic Interpretability of Large Language Models](https://mlsanity.com/paper/2505.03368/) (2025)
- [CLIP the Landscape: Automated Tagging of Crowdsourced Landscape Images](https://mlsanity.com/paper/2506.12214/) (2025)
- [Much of Geospatial Web Search Is Beyond Traditional GIS](https://mlsanity.com/paper/2605.11336/) (2026)

## Abstract (arXiv)

Sentence transformers are language models designed to perform semantic search. This study investigates the capacity of sentence transformers, fine-tuned on general question-answering datasets for asymmetric semantic search, to associate descriptions of human-generated routes across Great Britain with queries often used to describe hiking experiences. We find that sentence transformers have some zero-shot capabilities to understand quasi-geospatial concepts, such as route types and difficulty, suggesting their potential utility for routing recommendation systems.