mlsanityData to 14 Oct 2026

See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Model

arXiv 2509.16087 ↗First posted 19 Sep 2025cs.CV cs.AIcited by 15

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We introduce SEE&TREK, the first training-free prompting framework tailored to enhance the spatial understanding of Multimodal Large Language Models (MLLMS) under vision-only constraints. While prior efforts have incorporated modalities like depth or point clouds to improve spatial reasoning, purely visualspatial understanding remains underexplored. SEE&TREK addresses this gap by focusing on two core principles: increasing visual diversity and motion reconstruction. For visual diversity, we conduct Maximum Semantic Richness Sampling, which employs an off-the-shell perception model to extract semantically rich keyframes that capture scene structure. For motion reconstruction, we simulate visual trajectories and encode relative spatial positions into keyframes to preserve both spatial relations and temporal coherence. Our method is training&GPU-free, requiring only a single forward pass, and can be seamlessly integrated into existing MLLM'S. Extensive experiments on the VSI-B ENCH and STI-B ENCH show that S EE &T REK consistently boosts various MLLM S performance across diverse spatial reasoning tasks with the most +3.5% improvement, offering a promising path toward stronger spatial intelligence.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 62 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

50 of 62 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?31 Mar
  2. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models27 Mar
  3. 2025ST-VLM: Kinematic Instruction Tuning for Spatio-Temporal Reasoning in Vision-Language Models25 Mar
  4. 20253DAxisPrompt: Promoting the 3D Grounding and Reasoning in GPT-4o17 Mar
  5. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models2 Jan
  6. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  7. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun

Other references · 43

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Segment Anythingcited by 8,234
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  9. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  10. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  11. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  12. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
31 more
  1. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  2. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
  3. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  4. 2025Kimi-VL Technical Reportcited by 388
  5. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  6. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  7. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  8. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  9. 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
  10. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 176
  11. 2025Adaptive Keyframe Sampling for Long Video Understandingcited by 167
  12. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 138
  13. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 132
  14. 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
  15. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 88
  16. 2025M-LLM Based Video Frame Selection for Efficient Video Understandingcited by 88
  17. 2025VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videoscited by 77
  18. 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 65
  19. 2024Omni6DPose: A Benchmark and Model for Universal 6D Object Pose Estimation and Trackingcited by 50
  20. 2025Generative Frame Sampler for Long Video Understandingcited by 37
  21. 2025Logic-in-Frames: Dynamic Keyframe Search via Visual Semantic-Logical Verification for Long Video Understandingcited by 34
  22. 2024Attention Prompting on Image for Large Vision-Language Modelscited by 23
  23. 2025QuoTA: Query-oriented Token Assignment via CoT Query Decouple for Long Video Comprehensioncited by 21
  24. 2025Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMscited by 19
  25. 2025Hybrid-Level Instruction Injection for Video Token Compression in Multi-modal Large Language Modelscited by 19
  26. 2024ReWind: Understanding Long Videos with Instructed Learnable Memorycited by 17
  27. 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understandingcited by 17
  28. 2025Memory-enhanced Retrieval Augmentation for Long Video Understandingcited by 17
  29. 2025∞-Video: A Training-Free Approach to Long Video Understanding via Continuous-Time Memory Consolidationcited by 14
  30. 2025Agentic Keyframe Search for Video Question Answeringcited by 13
  31. 2025Breaking the Encoder Barrier for Seamless Video-Language Understandingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 14.

15 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models
  2. 2025Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture
  3. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence
  4. 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
  5. 2026GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs
  6. 2026The Spatial Blindspot of Vision-Language Models
  7. 2026Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models
  8. 2026ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?