mlsanityData to 7 Oct 2026

Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training

arXiv 2603.12255 ↗First posted 12 Mar 2026cs.CV cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Humans perceive and understand real-world spaces through a stream of visual observations. Therefore, the ability to streamingly maintain and update spatial evidence from potentially unbounded video streams is essential for spatial intelligence. The core challenge is not simply longer context windows but how spatial information is selected, organized, and retained over time. In this paper, we propose Spatial-TTT towards streaming visual-based spatial intelligence with test-time training (TTT), which adapts a subset of parameters (fast weights) to capture and organize spatial evidence over long-horizon scene videos. Specifically, we design a hybrid architecture and adopt large-chunk updates parallel with sliding-window attention for efficient spatial video processing. To further promote spatial awareness, we introduce a spatial-predictive mechanism applied to TTT layers with 3D spatiotemporal convolution, which encourages the model to capture geometric correspondence and temporal continuity across frames. Beyond architecture design, we construct a dataset with dense 3D spatial descriptions, which guides the model to update its fast weights to memorize and organize global 3D spatial signals in a structured manner. Extensive experiments demonstrate that Spatial-TTT improves long-horizon spatial understanding and achieves state-of-the-art performance on video spatial benchmarks. Project page: https://liuff19.github.io/Spatial-TTT.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

56 of 66 references.

References · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  10. 2025Qwen3-VL Technical Reportcited by 2,534
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  12. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
38 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  4. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  5. 2022Planning-oriented Autonomous Drivingcited by 1,068
  6. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  7. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  8. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  9. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  10. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  11. 2024Long Context Transfer from Language to Visioncited by 502
  12. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  13. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  14. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
  15. 2025Seed1.5-VL Technical Reportcited by 367
  16. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  17. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  18. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  19. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  20. 2024Titans: Learning to Memorize at Test Timecited by 303
  21. 2022Test-Time Training with Masked Autoencoderscited by 208
  22. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
  23. 2016Using Fast Weights to Attend to the Recent Pastcited by 161
  24. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
  25. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
  26. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 154
  27. 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 152
  28. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  29. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 130
  30. 2025Test-Time Training Done Rightcited by 121
  31. 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 91
  32. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 88
  33. 2025Visual Spatial Tuningcited by 85
  34. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
  35. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 74
  36. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
  37. 2025Test-time regression: a unifying framework for designing sequence models with associative memorycited by 63
  38. 2020Tent: Fully Test-time Adaptation by Entropy Minimizationcited by 59

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.