# HeatTok: Enhancing Remote Sensing Image Understanding via Thermodiffusion-based Tokenization

arXiv: https://arxiv.org/abs/2608.22485 · HTML: https://mlsanity.com/paper/2608.22485/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 12 of its 56 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 12 of 12)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [Vision Transformer with Super Token Sampling](https://mlsanity.com/paper/2211.11167/) (2022)
- [SegEarth-R1: Geospatial Pixel Reasoning via Large Language Model](https://mlsanity.com/paper/2504.09644/) (2025)
- [Subobject-level Image Tokenization](https://mlsanity.com/paper/2402.14327/) (2024)
- [Superpixel Tokenization for Vision Transformers: Preserving Semantic Integrity in Visual Tokens](https://mlsanity.com/paper/2412.04680/) (2024)
- [IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment](https://mlsanity.com/paper/2607.12375/) (2026)

## Abstract (arXiv)

Current visual tokenizers in Multimodal Large Language Models (MLLMs) predominantly rely on patch-based partitioning, which causes severe semantic mixture and object fragmentation in remote sensing imagery due to the irregular contours of geo-objects. Moreover, existing adaptive methods struggle to extract precise object-level tokens and lack dedicated geometric positional encodings for irregular regions. In this paper, we propose HeatTok, a semantic-aware tokenizer driven by thermodiffusion aggregation. Inspired by the physical principles of heat conduction, HeatTok adaptively merges adjacent homogeneous regions to generate semantically independent, object-aligned irregular tokens. To enable MLLMs to perceive these irregular shapes, we design the Gaussian Multimodal Rotary Positional Embedding (G-MRoPE), which models token spatial distributions via 2D Gaussians and explicitly injects center, scale, and orientation cues. Extensive evaluations on the VRSBench and EarthVQA datasets demonstrate that HeatTok effectively preserves object-level semantic integrity and achieves state-of-the-art performance under a reasonable token budget. The code is available: https://github.com/YingyingYan1/HeatTok.