Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training
Not on the map. Why?
Abstract (arXiv)
Humans perceive and understand real-world spaces through a stream of visual observations. Therefore, the ability to streamingly maintain and update spatial evidence from potentially unbounded video streams is essential for spatial intelligence. The core challenge is not simply longer context windows but how spatial information is selected, organized, and retained over time. In this paper, we propose Spatial-TTT towards streaming visual-based spatial intelligence with test-time training (TTT), which adapts a subset of parameters (fast weights) to capture and organize spatial evidence over long-horizon scene videos. Specifically, we design a hybrid architecture and adopt large-chunk updates parallel with sliding-window attention for efficient spatial video processing. To further promote spatial awareness, we introduce a spatial-predictive mechanism applied to TTT layers with 3D spatiotemporal convolution, which encourages the model to capture geometric correspondence and temporal continuity across frames. Beyond architecture design, we construct a dataset with dense 3D spatial descriptions, which guides the model to update its fast weights to memorize and organize global 3D spatial signals in a structured manner. Extensive experiments demonstrate that Spatial-TTT improves long-horizon spatial understanding and achieves state-of-the-art performance on video spatial benchmarks. Project page: https://liuff19.github.io/Spatial-TTT.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
56 of 66 references.
References · 56
- 2017Attention Is All You Needcited by 59,932
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
38 more
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
- 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
- 2024Long Context Transfer from Language to Visioncited by 502
- 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
- 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
- 2025Seed1.5-VL Technical Reportcited by 367
- 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
- 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
- 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
- 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
- 2024Titans: Learning to Memorize at Test Timecited by 303
- 2022Test-Time Training with Masked Autoencoderscited by 208
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
- 2016Using Fast Weights to Attend to the Recent Pastcited by 161
- 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
- 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
- 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 154
- 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 152
- 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
- 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 130
- 2025Test-Time Training Done Rightcited by 121
- 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 91
- 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 88
- 2025Visual Spatial Tuningcited by 85
- 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
- 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 74
- 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
- 2025Test-time regression: a unifying framework for designing sequence models with associative memorycited by 63
- 2020Tent: Fully Test-time Adaptation by Entropy Minimizationcited by 59
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026WAM-TTT: Steering World-Action Models by Watching Human Play at Test Timecited by 6
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 6
- 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 4
- 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMscited by 3
- 2026Towards Spatial Supersensing in the Wildcited by 2
- 2026TTTIR: Unlocking Instance-Specific State Evolution via Test-Time Training for Image Restorationcited by 0
- 2026When Should the Count Change? Learning State Maintenance for Causal Video Countingcited by 0
- 2026Uruqi: Learning Spatial Cognition from Visual Experiencecited by 0
- 2026Uruqi: Learning Spatial Cognition from Visual Experience30 Sep 2026
- 2026When Should the Count Change? Learning State Maintenance for Causal Video Counting28 Sep 2026
- 2026TTTIR: Unlocking Instance-Specific State Evolution via Test-Time Training for Image Restoration15 Sep 2026
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning27 Aug 2026
- 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs13 Aug 2026
- 2026Towards Spatial Supersensing in the Wild15 Jul 2026
- 2026WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time8 Jul 2026
- 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs2 Jun 2026