mlsanityData to 14 Oct 2026

TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies

arXiv 2412.10345 ↗First posted 13 Dec 2024cs.RO cs.AIcited by 304

On the map: Vision-language-action models → 3D and spatial-aware VLA

Abstract (arXiv)

Although large vision-language-action (VLA) models pretrained on extensive robot datasets offer promising generalist policies for robotic learning, they still struggle with spatial-temporal dynamics in interactive robotics, making them less effective in handling complex tasks, such as manipulation. In this work, we introduce visual trace prompting, a simple yet effective approach to facilitate VLA models' spatial-temporal awareness for action prediction by encoding state-action trajectories visually. We develop a new TraceVLA model by finetuning OpenVLA on our own collected dataset of 150K robot manipulation trajectories using visual trace prompting. Evaluations of TraceVLA across 137 configurations in SimplerEnv and 4 tasks on a physical WidowX robot demonstrate state-of-the-art performance, outperforming OpenVLA by 10% on SimplerEnv and 3.5x on real-robot tasks and exhibiting robust generalization across diverse embodiments and scenarios. To further validate the effectiveness and generality of our method, we present a compact VLA model based on 4B Phi-3-Vision, pretrained on the Open-X-Embodiment and finetuned on our dataset, rivals the 7B OpenVLA baseline while significantly improving inference efficiency.

Key facts. Cited by 304 papers in the map (data through 14 Oct 2026). Placed in 3D and spatial-aware VLA, one of 13 directions in Vision-language-action models. Builds on 42 of its 46 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

42 of 46 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  12. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
30 more
  1. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  2. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  3. 2022A Generalist Agentcited by 808
  4. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  5. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  6. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  7. 2023CoTracker: It is Better to Track Togethercited by 522
  8. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 339
  9. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
  10. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
  11. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  12. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  13. 2023Any-point Trajectory Modeling for Policy Learningcited by 215
  14. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 190
  15. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 169
  16. 2023Language-Driven Representation Learning for Roboticscited by 165
  17. 2023Vision-Language Models as Success Detectorscited by 142
  18. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigationcited by 137
  19. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 133
  20. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 124
  21. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 90
  22. 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learningcited by 87
  23. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 57
  24. 2024LLaRA: Supercharging Robot Learning Data for Vision-Language Policycited by 52
  25. 2024Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasetscited by 23
  26. 2024PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Controlcited by 21
  27. 2023Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Trainingcited by 18
  28. 2024Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusioncited by 14
  29. 2024Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Losscited by 10
  30. 2024ARDuP: Active Region Video Diffusion for Universal Policiescited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 118; 2026: 185.

304 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  2. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
  3. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  4. 2026World Action Models are Zero-shot Policiescited by 465
  5. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 410
  6. 2026Causal World Modeling for Robot Controlcited by 321
  7. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 303
  8. 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
  9. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 213
  10. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 197
  11. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 192
  12. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 156
38 more
  1. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 145
  2. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 139
  3. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
  4. 2025Unified Vision-Language-Action Modelcited by 134
  5. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 134
  6. 2025NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Taskscited by 127
  7. 2025FLARE: Robot Learning with Implicit World Modelingcited by 113
  8. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 108
  9. 2025UP-VLA: A Unified Understanding and Prediction Model for Embodied Agentcited by 102
  10. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  11. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  12. 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 74
  13. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 72
  14. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 69
  15. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 67
  16. 2025MemER: Scaling Up Memory for Robot Control via Experience Retrievalcited by 67
  17. 2025Training-Time Action Conditioning for Efficient Real-Time Chunkingcited by 64
  18. 2025villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Modelscited by 63
  19. 2026MEM: Multi-Scale Embodied Memory for Vision Language Action Modelscited by 61
  20. 2025Hume: Introducing System-2 Thinking in Visual-Language-Action Modelcited by 56
  21. 20254D-VLA: Spatiotemporal Vision-Language-Action Pretraining with Cross-Scene Calibrationcited by 56
  22. 2025Training Strategies for Efficient Embodied Reasoningcited by 55
  23. 2025FlowVLA: Visual Chain of Thought-based Motion Reasoning for Vision-Language-Action Modelscited by 50
  24. 2026MolmoAct2: Action Reasoning Models for Real-world Deploymentcited by 50
  25. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 49
  26. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  27. 2026VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Modelscited by 39
  28. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Reviewcited by 38
  29. 2025Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulationcited by 38
  30. 2025VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Modelscited by 36
  31. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulationcited by 35
  32. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Contextcited by 35
  33. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  34. 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Modelscited by 33
  35. 2025Adversarial Attacks on Robotic Vision Language Action Modelscited by 32
  36. 2025HiF-VLA: Hindsight, Insight and Foresight through Motion Representation for Vision-Language-Action Modelscited by 32
  37. 2024CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordancecited by 31
  38. 2025EO-1: An Open Unified Embodied Foundation Model for General Robot Controlcited by 31

50 most cited of 304 All 304 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks
  2. 2025MAP-VLA: Memory-Augmented Prompting for Vision-Language-Action Model in Robotic Manipulation
  3. 2026Colosseum V2: Benchmarking Generalization for Vision-Language-Action Models
  4. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
  5. 2026ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation
  6. 2026FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies
  7. 2025VideoVLA: Video Generators Can Be Generalizable Robot Manipulators
  8. 2026DiMaS: Distribution Matching for Steering Vision-Language-Action Models