mlsanityData to 7 Oct 2026

SeeQ: Training Generalist Value Functions for Long-Horizon Robotic Manipulation

arXiv 2609.22085 ↗First posted 18 Sep 2026cs.ROcited by 0

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

Despite rapid progress, generalist robot policies remain brittle on complex, long-horizon tasks that comprise multiple stages or require repeated attempts and deliberation on the same underlying stage before success. Q-value functions can improve these policies by ranking candidate actions or guiding policy improvement, but learning from sparse task-level rewards entails long credit-assignment horizons, difficult Bellman backups, and broad data-coverage requirements. We introduce SeeQ (Subtask-elicited Q-functions), which instead learns Q-values for the currently active subtask. This shortens the value-prediction horizon and enables effective learning with temporal-difference (TD) objectives. During training, subtask-level annotations present in offline robot data provide the decomposition and enable learning from broad, potentially suboptimal robot datasets. To eliminate the need for human annotations or modular subtask prediction systems at test time, our Q-function architecture is trained to autoregressively predict the active subtask in natural language before estimating its value. We instantiate SeeQ using a base vision-language backbone, pretrain it on diverse open-source robot manipulation data, and finetune it on downstream tasks. Across four real-world manipulation tasks on two bimanual robot platforms, the SeeQ value function substantially improves best-of-N policy steering.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 29 of its 41 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

29 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning31 Jul
  2. 2026Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies1 May
  3. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance17 Oct

Other references · 26

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  7. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  8. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  9. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  10. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
  11. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  12. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
14 more
  1. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  2. 2025Reinforcement Learning with Action Chunkingcited by 97
  3. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  4. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  5. 2025Horizon Reduction Makes RL Scalablecited by 55
  6. 2023HYDRA: Hybrid Robot Actions for Imitation Learningcited by 54
  7. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  8. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 49
  9. 2025RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correctioncited by 35
  10. 2023Robotic Offline RL from Internet Videos via Value-Function Pre-Trainingcited by 31
  11. 2024Offline Actor-Critic Reinforcement Learning Scales to Large Modelscited by 30
  12. 2023Universal Visual Decomposer: Long-Horizon Manipulation Made Easycited by 23
  13. 2026Demystifying Action Space Design for Robotic Manipulation Policiescited by 9
  14. 2026RynnValue: Scaling Robotic Value Foundation Models with Temporal Distancecited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Generalizable Dense Reward for Long-Horizon Robotic Tasks
  2. 2026τ0-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation
  3. 2024Vision Language Models are In-Context Value Learners
  4. 2026World Value Models for Robotic Manipulation
  5. 2025Training-free Generation of Temporally Consistent Rewards from VLMs
  6. 2026AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models
  7. 2025LoLA: Long Horizon Latent Action Learning for General Robot Manipulation
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Control