# Enabling Intrinsic Reasoning over Dense Geospatial Embeddings with DFR-Gemma

arXiv: https://arxiv.org/abs/2604.07490 · HTML: https://mlsanity.com/paper/2604.07490/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 8 Apr 2026.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 11 of 11)

- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024)
- [Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning](https://mlsanity.com/paper/2502.03275/) (2025)
- [Continuous Chain of Thought Enables Parallel Exploration and Reasoning](https://mlsanity.com/paper/2505.23648/) (2025)
- [General Geospatial Inference with a Population Dynamics Foundation Model](https://mlsanity.com/paper/2411.07207/) (2024)
- [Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questions](https://mlsanity.com/paper/2502.18470/) (2025)
- [A systematic review of geospatial location embedding approaches in large language models: A path to spatial AI systems](https://mlsanity.com/paper/2401.10279/) (2024)
- [Deploying Geospatial Foundation Models in the Real World: Lessons from WorldCereal](https://mlsanity.com/paper/2508.00858/) (2025)
- [Attention Prompt Tuning: Parameter-efficient Adaptation of Pre-trained Models for Spatiotemporal Modeling](https://mlsanity.com/paper/2403.06978/) (2024)
- [HAMburger: Accelerating LLM Inference via Token Smashing](https://mlsanity.com/paper/2505.20438/) (2025)

## Abstract (arXiv)

Representation learning for geospatial and spatio-temporal data plays a critical role in enabling general-purpose geospatial intelligence. Recent geospatial foundation models, such as the Population Dynamics Foundation Model (PDFM), encode complex population and mobility dynamics into compact embeddings. However, their integration with Large Language Models (LLMs) remains limited. Existing approaches to LLM integration treat these embeddings as retrieval indices or convert them into textual descriptions for reasoning, introducing redundancy, token inefficiency, and numerical inaccuracies. We propose Direct Feature Reasoning-Gemma (DFR-Gemma), a novel framework that enables LLMs to reason directly over dense geospatial embeddings. DFR aligns high-dimensional embeddings with the latent space of an LLM via a lightweight projector, allowing embeddings to be injected as semantic tokens alongside natural language instructions. This design eliminates the need for intermediate textual representations and enables intrinsic reasoning over spatial features. To evaluate this paradigm, we introduce a multi-task geospatial benchmark that pairs embeddings with diverse question-answer tasks, including feature querying, comparison, and semantic description. Experimental results show that DFR allows LLMs to decode latent spatial patterns and perform accurate zero-shot reasoning across tasks, while significantly improving efficiency compared to text-based baselines. Our results demonstrate that treating embeddings as primary data inputs, provides a more direct, efficient, and scalable approach to multimodal geospatial intelligence.