mlsanityData to 7 Oct 2026

Towards Flexible Inference in Sequential Decision Problems via Bidirectional Transformers

arXiv 2204.13326 ↗First posted 28 Apr 2022cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks. In this work, we observe that the same idea also applies naturally to sequential decision making, where many well-studied tasks like behavior cloning, offline RL, inverse dynamics, and waypoint conditioning correspond to different sequence maskings over a sequence of states, actions, and returns. We introduce the FlexiBiT framework, which provides a unified way to specify models which can be trained on many different sequential decision making tasks. We show that a single FlexiBiT model is simultaneously capable of carrying out many tasks with performance similar to or better than specialized models. Additionally, we show that performance can be further improved by fine-tuning our general model on specific tasks of interest.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 28 Apr 2022.

Builds on

26 of 36 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  9. 2018World Modelscited by 1,444
  10. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  11. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  12. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
14 more
  1. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  2. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  3. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  4. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  5. 2019Goal-conditioned Imitation Learningcited by 159
  6. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  7. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  8. 2020PMI-Masking: Principled masking of correlated spanscited by 54
  9. 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 53
  10. 2019Preferences Implicit in the State of the Worldcited by 33
  11. 2019A Generalized Framework of Sequence Generation with Application to Undirected Sequence Modelscited by 17
  12. 2021Teaching Autoregressive Language Models Complex Tasks By Demonstrationcited by 15
  13. 2019Learning Calibratable Policies using Programmatic Style-Consistencycited by 11
  14. 2021Assistive Tele-op: Leveraging Transformers to Collect Robotic Task Demonstrationscited by 11

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 2.

6 papers.

  1. 2022Deep Transformer Q-Networks for Partially Observable Reinforcement Learningcited by 35
  2. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  3. 2023Large Sequence Models for Sequential Decision-Making: A Surveycited by 19
  4. 2023Skill Transformer: A Monolithic Policy for Mobile Manipulationcited by 15
  5. 2024MobilityDL: A Review of Deep Learning From Trajectory Datacited by 6
  6. 2024Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systemscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2022UniMASK: Unified Inference in Sequential Decision Problems
  2. 2025On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inference
  3. 2024Efficient Sequential Decision Making with Large Language Models
  4. 2020Learning to Generalize for Sequential Decision Making
  5. 2026Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inference
  6. 2025From Next Token Prediction to (STRIPS) World Models
  7. 2026Distilling Sequential Computation in Transformer Language Models
  8. 2025Unified token representations for sequential decision models