mlsanityData to 14 Oct 2026

Task Tokens: A Flexible Approach to Adapting Behavior Foundation Models

arXiv 2503.22886 ↗First posted 28 Mar 2025cs.LG cs.ROcited by 3

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Recent advancements in imitation learning have led to transformer-based behavior foundation models (BFMs) that enable multi-modal, human-like control for humanoid agents. While excelling at zero-shot generation of robust behaviors, BFMs often require meticulous prompt engineering for specific tasks, potentially yielding suboptimal results. We introduce "Task Tokens", a method to effectively tailor BFMs to specific tasks while preserving their flexibility. Our approach leverages the transformer architecture of BFMs to learn a new task-specific encoder through reinforcement learning, keeping the original BFM frozen. This allows incorporation of user-defined priors, balancing reward design and prompt engineering. By training a task encoder to map observations to tokens, used as additional BFM inputs, we guide performance improvement while maintaining the model's diverse control characteristics. We demonstrate Task Tokens' efficacy across various tasks, including out-of-distribution scenarios, and show their compatibility with other prompting modalities. Our results suggest that Task Tokens offer a promising approach for adapting BFMs to specific control tasks while retaining their generalization capabilities.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 29 references that are in the map. First posted on arXiv 28 Mar 2025.

Builds on

18 of 29 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  7. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  8. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  9. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 218
  10. 2023Universal Humanoid Motion Representations for Physics-Based Controlcited by 116
  11. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 116
  12. 2021Learning One Representation to Optimize All Rewardscited by 115
6 more
  1. 2023CALM: Conditional Adversarial Latent Models for Directable Virtual Characterscited by 94
  2. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 88
  3. 2024Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Controlcited by 51
  4. 2024CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamicscited by 44
  5. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 42
  6. 2024SMPLOlympics: Sports Environments for Physically Simulated Humanoidscited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactionscited by 11
  2. 2026Zero-Shot Off-Policy Learningcited by 2
  3. 2026Progressively Learning Heterogeneous Skills in a Unified Latent Spacecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Prompt, Plan, Perform: LLM-based Humanoid Control via Quantized Imitation Learning
  2. 2026When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisited
  3. 2025BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning
  4. 2026I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning
  5. 2026Task-Centric Policy Optimization from Misaligned Motion Priors
  6. 2026Generalizing from References using a Multi-Task Reference and Goal-Driven RL Framework
  7. 2024In-Context Imitation Learning via Next-Token Prediction
  8. 2026Test-Time Compute for Humanoid Behavior Foundation Models