mlsanityData to 14 Oct 2026

UniTrackPLA: Unified Panorama-Language-Action Model for Instruction-Guided Navigation and Dynamic Person Tracking

arXiv 2610.00878 ↗First posted 1 Oct 2026cs.RO cs.CV eess.IVcited by 0

Not on the map. Why?

Abstract (arXiv)

General-purpose embodied robots should support both navigation toward language-specified destinations and dynamic person tracking under arbitrary initial target azimuths. However, existing methods typically rely on forward-facing observations and address these tasks with separate policies, limiting omnidirectional perception and unified closed-loop control. We present UniTrackPLA, a unified panorama-language-action model for instruction-guided navigation and dynamic person tracking. Its Panoramic-Aware Encoding (PAE) preserves the temporal and azimuthal structure of perspective views projected from each panorama, enabling perspective-pretrained visual encoders to process omnidirectional observations. A shared vision-language backbone grounds instructions in the panoramic context and predicts continuous robot-centric waypoint chunks for both tasks. World-Action Consistency (WAC) further predicts action-conditioned future visual states and verifies waypoint prefixes online, allowing reliable actions to be reused while triggering replanning upon inconsistency. We also introduce OmniTrackNav-Bench, comprising 5,000 simulated tracking trajectories, 10,000 simulated VLN routes, and 96 verified real-world routes, providing 919,978 waypoint-supervision instances. UniTrackPLA improves overall tracking SR from 23.50% to 35.00% and Omni-VLN SR/SPL from 13.00%/12.77% to 19.75%/19.29%. Incorporating 76 real-world routes further improves held-out [email protected] from 42.92% to 92.08%. Closed-loop experiments on a Go2-W robot demonstrate unified panoramic tracking and navigation across indoor and outdoor environments. The project page is at https://tw5775.github.io/UniTrackPLA.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 36 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

32 of 36 references.

References · 32

  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  2. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  3. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  4. 2025Embodied Navigation Foundation Modelcited by 72
  5. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 47
  6. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 43
  7. 2022Towards Versatile Embodied Navigationcited by 26
  8. 2025NavBench: Probing Multimodal Large Language Models for Embodied Navigationcited by 24
  9. 2024Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RLcited by 23
  10. 2025TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Trackingcited by 17
  11. 2025Omnidirectional Multi-Object Trackingcited by 13
  12. 2025PanoNav: Mapless Zero-Shot Object Navigation with Panoramic Scene Parsing and Dynamic Memorycited by 9
20 more
  1. 2025Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigationcited by 8
  2. 2025VLM Can Be a Good Assistant: Enhancing Embodied Visual Tracking with Self-Improving Vision-Language Modelscited by 7
  3. 2025PanoGen++: Domain-Adapted Text-Guided Panoramic Environment Generation for Vision-and-Language Navigationcited by 6
  4. 2026CoMaTrack: Competitive Multi-Agent Game-Theoretic Tracking with Vision-Language-Action Modelscited by 5
  5. 2025TPT-Bench: A Large-Scale, Long-Term and Robot-Egocentric Dataset for Benchmarking Target Person Trackingcited by 4
  6. 2025Hierarchical Instruction-aware Embodied Visual Trackingcited by 4
  7. 2026OmniVLN: Omnidirectional 3D Perception and Token-Efficient LLM Reasoning for Visual-Language Navigation across Air and Ground Platformscited by 4
  8. 2026Learning Panorama-Aware VLA for Mobile Manipulation with Whole-Body Teleoperationcited by 4
  9. 2025UrbanVLA: A Vision-Language-Action Model for Urban Micromobilitycited by 3
  10. 2026Navigation beyond Wayfinding: Robots Collaborating with Visually Impaired Users for Environmental Interactionscited by 3
  11. 2026AdaTracker: Learning Adaptive In-Context Policy for Cross-Embodiment Active Visual Trackingcited by 3
  12. 2026Instance-level Visual Active Tracking with Occlusion-Aware Planningcited by 3
  13. 2026EAGOR: Embodied Reasoning in Omni-directioncited by 3
  14. 2026ReferTrack: Referring Then Tracking for Embodied Visual Trackingcited by 3
  15. 2026OmniDP: Beyond-FOV Large-Workspace Humanoid Manipulation with Omnidirectional 3D Perceptioncited by 2
  16. 2026P2DNav: Panorama-to-Downview Reasoning for Zero-shot Vision-and-Language Navigationcited by 2
  17. 2024Reflex-Based Open-Vocabulary Navigation without Prior Knowledge Using Omnidirectional Camera and Multiple Vision-Language Modelscited by 1
  18. 2026UESF-Bench: Benchmarking and Probing for Unified Embodied Seeking and Followingcited by 1
  19. 2026SC2-WM: A Self-Correcting World Model with Closed-Loop Feedback for Vision-and-Language Navigation in Continuous Environmentscited by 1
  20. 2026360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agentscited by 1

Built on by

Not cited yet.