# Logically Constrained Robotics Transformers for Enhanced Perception-Action Planning

arXiv: https://arxiv.org/abs/2408.05336 · HTML: https://mlsanity.com/paper/2408.05336/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 9 Aug 2024.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [ChatGPT for Robotics: Design Principles and Model Abilities](https://mlsanity.com/paper/2306.17582/) (2023)
- [A Survey on Vision-Language-Action Models for Embodied AI](https://mlsanity.com/paper/2405.14093/) (2024)
- [Q-Learning for Robust Satisfaction of Signal Temporal Logic Specifications](https://mlsanity.com/paper/1609.07409/) (2016)
- [NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Models](https://mlsanity.com/paper/2305.07766/) (2023)
- [Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methods](https://mlsanity.com/paper/2008.00097/) (2020)
- [SMART: Self-supervised Multi-task pretrAining with contRol Transformers](https://mlsanity.com/paper/2301.09816/) (2023)
- [Learning Safe, Generalizable Perception-based Hybrid Control with Certificates](https://mlsanity.com/paper/2201.00932/) (2022)
- [PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training](https://mlsanity.com/paper/2209.11133/) (2022)
- [Model-based Reinforcement Learning from Signal Temporal Logic Specifications](https://mlsanity.com/paper/2011.04950/) (2020)
- [Specification Patterns for Robotic Missions](https://mlsanity.com/paper/1901.02077/) (2019)
- [Mixed-Integer Programming for Signal Temporal Logic with Fewer Binary Variables](https://mlsanity.com/paper/2204.06367/) (2022)
- [Reactive and Safe Road User Simulations using Neural Barrier Certificates](https://mlsanity.com/paper/2109.06689/) (2021)
- [Follow The Rules: Online Signal Temporal Logic Tree Search for Guided Imitation Learning in Stochastic Domains](https://mlsanity.com/paper/2209.13737/) (2022)
- [Safe Planning through Incremental Decomposition of Signal Temporal Logic Specifications](https://mlsanity.com/paper/2403.10554/) (2024)
- [Challenges in Close-Proximity Safe and Seamless Operation of Manned and Unmanned Aircraft in Shared Airspace](https://mlsanity.com/paper/2211.06932/) (2022)
- [ConBaT: Control Barrier Transformer for Safe Policy Learning](https://mlsanity.com/paper/2303.04212/) (2023)

## Built on by (most cited first, 4 of 4)

- [Explain Like I'm Five: Using LLMs to Improve PDE Surrogate Models with Text](https://mlsanity.com/paper/2410.01137/) (2024)
- [Constrained Decoding for Safe Robot Navigation Foundation Models](https://mlsanity.com/paper/2509.01728/) (2025)
- [hint2: Hierarchical World Models for Inference-Time Temporal Logic Guidance](https://mlsanity.com/paper/2608.13678/) (2026)
- [Graded Transformers](https://mlsanity.com/paper/2507.20108/) (2025)

## Abstract (arXiv)

With the advent of large foundation model based planning, there is a dire need to ensure their output aligns with the stakeholder's intent. When these models are deployed in the real world, the need for alignment is magnified due to the potential cost to life and infrastructure due to unexpected faliures. Temporal Logic specifications have long provided a way to constrain system behaviors and are a natural fit for these use cases. In this work, we propose a novel approach to factor in signal temporal logic specifications while using autoregressive transformer models for trajectory planning. We also provide a trajectory dataset for pretraining and evaluating foundation models. Our proposed technique acheives 74.3 % higher specification satisfaction over the baselines.