mlsanityData to 14 Oct 2026

EgoAlign: Situated 3D Understanding without Situation Descriptions

arXiv 2610.16176 ↗First posted 12 Oct 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Situated 3D Question Answering requires models to interpret a 3D scene relative to an agent's pose. Existing methods commonly convey that pose through a natural-language situation description, which may be unavailable or costly to obtain in real-world deployments. While some methods additionally incorporate numerical pose, we find that randomly shuffling the agent pose causes little performance degradation, indicating limited functional dependence on the numerical pose. To address this gap, we introduce EgoAlign, a simple 3D language model that represents a scene in an agent-centric coordinate frame, together with a dedicated egocentric alignment stage comprising four complementary tasks: object category recognition, egocentric direction prediction, metric distance estimation, and spatial relation prediction. This stage teaches the model to consistently interpret scene geometry relative to the agent's pose before task-specific fine-tuning. EgoAlign achieves the strongest performance on SQA3D, Real-3DQA, and MSQA among compared methods while remaining highly efficient. More importantly, removing textual situation descriptions yields comparable or better downstream performance, whereas shuffling EgoAlign's numerical pose causes substantial degradation. These results show that EgoAlign establishes agent-centric spatial context directly from numerical pose without requiring textual situation descriptions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 31 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

28 of 31 references.

References · 28

  1. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  2. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  3. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
  4. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  5. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  6. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  7. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 210
  8. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
  9. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
  10. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  11. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 138
  12. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 129
16 more
  1. 2024Grounded 3D-LLM with Referent Tokenscited by 115
  2. 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 110
  3. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 109
  4. 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformercited by 68
  5. 2024Multi-modal Situated Reasoning in 3D Scenescited by 51
  6. 2025Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representationscited by 45
  7. 2024LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferencescited by 38
  8. 2024Situational Awareness Matters in 3D Vision Language Reasoningcited by 33
  9. 2024SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Modelscited by 27
  10. 2025Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysiscited by 23
  11. 2024PerLA: Perceptive 3D Language Assistantcited by 9
  12. 2026Do 3D Large Language Models Really Understand 3D Spatial Relationships?cited by 9
  13. 2025SceneCOT: Eliciting Grounded Chain-of-Thought Reasoning in 3D Scenescited by 8
  14. 2025Empowering Large Language Models with 3D Situation Awarenesscited by 6
  15. 2026Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understandingcited by 5
  16. 2026Direction-aware 3D Large Multimodal Modelscited by 1

Built on by

Not cited yet.