mlsanityData to 7 Oct 2026

FineART: Fine-Grained Annotated Robotic Trajectory Dataset and Vision-Language-Action Model for Bimanual Manipulation

arXiv 2609.36416 ↗First posted 29 Sep 2026cs.RO cs.AI cs.CVcited by 0

On the map: Vision-language-action models → Human-video and latent-action pretraining

Abstract (arXiv)

Robots operating in real-world environments must often execute complex, multi-step bimanual tasks over long horizons rather than single, isolated actions. Current manipulation datasets provide limited support for this capability: although single-arm datasets reach hundreds of thousands of trajectories, they typically provide only one high-level instruction per episode, while existing bimanual datasets provide subtask annotations for only part of their recorded hours. We present FineART, a densely annotated bimanual manipulation dataset comprising 40,543 episodes (1,718 hours) and 533,913 subtasks across 151 tasks. We also introduce FineART-VLA, a vision-language-action policy that predicts its own next subtask to guide its actions. Mid-training on FineART's subtask annotations improves FineART-VLA's success at following spatial instructions from 32.0% to 100.0%. With step-by-step human subtask guidance, success on unseen long-horizon tasks increases from 16.0% to 76.0%. Furthermore, FineART-VLA matches baseline performance on a new robot with 10x less fine-tuning data and generalizes zero-shot to unseen tasks. We open-source the full dataset, model weights, and training code.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Human-video and latent-action pretraining, one of 13 directions in Vision-language-action models. Builds on 29 of its 34 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

29 of 34 references.

References · 29

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  7. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  9. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  10. 2017Dense-Captioning Events in Videoscited by 789
  11. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  12. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
17 more
  1. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  2. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  3. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  4. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
  5. 2024RT-H: Action Hierarchies Using Languagecited by 204
  6. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  7. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  8. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  9. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  10. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  11. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
  12. 2026LeRobot: An Open-Source Library for End-to-End Robot Learningcited by 70
  13. 2026MolmoAct2: Action Reasoning Models for Real-world Deploymentcited by 45
  14. 2025SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulationcited by 36
  15. 2024Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Modelscited by 14
  16. 2026SADP: Subgoal-Aware Diffusion Policy for Long-Horizon Manipulation Learned from Foundation Model Generated Demonstrationscited by 1
  17. 2026How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025TwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Models
  2. 2026RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation
  3. 2024PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks
  4. 2025Towards a Generalizable Bimanual Foundation Policy via Flow-based Video Prediction
  5. 2026Scaling Bimanual Household Manipulation from 1,500 hours of Demonstrations to On-Policy Corrections
  6. 2026Semantic-Geometric Task Representations for Bimanual Manipulation from Human Demonstrations to Robot Action Planning
  7. 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulation
  8. 2026See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation