# Advancing Complex Wide-Area Scene Understanding with Hierarchical Coresets Selection

arXiv: https://arxiv.org/abs/2507.13061 · HTML: https://mlsanity.com/paper/2507.13061/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 75 references that are in the map.
- First posted on arXiv 17 Jul 2025.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [VAD: Vectorized Scene Representation for Efficient Autonomous Driving](https://mlsanity.com/paper/2303.12077/) (2023)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm](https://mlsanity.com/paper/2110.05208/) (2021)
- [AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification](https://mlsanity.com/paper/1608.05167/) (2016)
- [Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning](https://mlsanity.com/paper/2403.11401/) (2024)
- [Deep Semantic Segmentation of Natural and Medical Images: A Review](https://mlsanity.com/paper/1910.07655/) (2019)
- [Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively](https://mlsanity.com/paper/2401.02955/) (2024)
- [Coresets and Sketches](https://mlsanity.com/paper/1601.00617/) (2016)
- [Max-Margin Token Selection in Attention Mechanism](https://mlsanity.com/paper/2306.13596/) (2023)
- [TrashCan: A Semantically-Segmented Dataset towards Visual Detection of Marine Debris](https://mlsanity.com/paper/2007.08097/) (2020)
- [Coresets for Near-Convex Functions](https://mlsanity.com/paper/2006.05482/) (2020)
- [Strong Coresets for k-Median and Subspace Approximation: Goodbye Dimension](https://mlsanity.com/paper/1809.02961/) (2018)
- [Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey](https://mlsanity.com/paper/1803.03352/) (2018)
- [Fast Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/1708.02550/) (2017)
- [TSP-Transformer: Task-Specific Prompts Boosted Transformer for Holistic Scene Understanding](https://mlsanity.com/paper/2311.03427/) (2023)
- [Towards Task Sampler Learning for Meta-Learning](https://mlsanity.com/paper/2307.08924/) (2023)
- [On Optimal Coreset Construction for Euclidean (k,z)-Clustering](https://mlsanity.com/paper/2211.11923/) (2022)
- [BLO-SAM: Bi-level Optimization Based Overfitting-Preventing Finetuning of SAM](https://mlsanity.com/paper/2402.16338/) (2024)
- [Integrating Deep Metric Learning with Coreset for Active Learning in 3D Segmentation](https://mlsanity.com/paper/2411.15763/) (2024)
- [Dynamic texture and scene classification by transferring deep image features](https://mlsanity.com/paper/1502.00303/) (2015)
- [ContextCLIP: Contextual Alignment of Image-Text pairs on CLIP visual representations](https://mlsanity.com/paper/2211.07122/) (2022)

## Built on by (most cited first, 1 of 1)

- [Unsupervised Multi-Attention Meta Transformer for Rotating Machinery Fault Diagnosis](https://mlsanity.com/paper/2509.09251/) (2025)

## Abstract (arXiv)

Scene understanding is one of the core tasks in computer vision, aiming to extract semantic information from images to identify objects, scene categories, and their interrelationships. Although advancements in Vision-Language Models (VLMs) have driven progress in this field, existing VLMs still face challenges in adaptation to unseen complex wide-area scenes. To address the challenges, this paper proposes a Hierarchical Coresets Selection (HCS) mechanism to advance the adaptation of VLMs in complex wide-area scene understanding. It progressively refines the selected regions based on the proposed theoretically guaranteed importance function, which considers utility, representativeness, robustness, and synergy. Without requiring additional fine-tuning, HCS enables VLMs to achieve rapid understandings of unseen scenes at any scale using minimal interpretable regions while mitigating insufficient feature density. HCS is a plug-and-play method that is compatible with any VLM. Experiments demonstrate that HCS achieves superior performance and universality in various tasks.