mlsanityData to 7 Oct 2026

SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention

arXiv 2312.01990 ↗First posted 4 Dec 2023cs.RO cs.AIcited by 20

Not on the map. Why?

Abstract (arXiv)

We present Self-Adaptive Robust Attention for Robotics Transformers (SARA-RT): a new paradigm for addressing the emerging challenge of scaling up Robotics Transformers (RT) for on-robot deployment. SARA-RT relies on the new method of fine-tuning proposed by us, called up-training. It converts pre-trained or already fine-tuned Transformer-based robotic policies of quadratic time complexity (including massive billion-parameter vision-language-action models or VLAs), into their efficient linear-attention counterparts maintaining high quality. We demonstrate the effectiveness of SARA-RT by speeding up: (a) the class of recently introduced RT-2 models, the first VLA robotic policies pre-trained on internet-scale data, as well as (b) Point Cloud Transformer (PCT) robotic policies operating on large point clouds. We complement our results with the rigorous mathematical analysis providing deeper insight into the phenomenon of SARA.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 33 of its 38 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

33 of 38 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2020Rethinking Attention with Performerscited by 1,584
  12. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
21 more
  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  2. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  3. 2020PCT: Point cloud transformercited by 730
  4. 2022Transformers learn in-context by gradient descentcited by 714
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  6. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  7. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  8. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  9. 2021Random Feature Attentioncited by 350
  10. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  11. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 291
  12. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  13. 2022cosFormer: Rethinking Softmax in Attentioncited by 212
  14. 2023Large Language Models as General Pattern Machinescited by 196
  15. 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
  16. 2018Simple random search provides a competitive approach to reinforcement learningcited by 75
  17. 2022i-Sim2Real: Reinforcement Learning of Robotic Policies in Tight Human-Robot Interaction Loopscited by 30
  18. 2023Efficient Attention via Control Variatescited by 23
  19. 2022Chefs' Random Tables: Non-Trigonometric Random Featurescited by 20
  20. 2023Robotic Table Tennis: A Case Study into a High Speed Learning Systemcited by 12
  21. 2023FAVOR#: Sharp Attention Kernel Approximations via New Classes of Positive Random Featurescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 9; 2026: 4.

20 papers.

Sort:

By area:

  1. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  2. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  3. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  4. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  5. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  6. 2024SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluationcited by 31
  7. 2025OvercookedV2: Rethinking Overcooked for Zero-Shot Coordinationcited by 14
  8. 2024Modeling the Real World with High-Density Visual Particle Dynamicscited by 12
  9. 2025RationalVLA: A Rational Vision-Language-Action Model with Dual Systemcited by 12
  10. 2024Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Controlcited by 11
  11. 2025CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulationcited by 7
  12. 2025Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action modelscited by 6
8 more
  1. 2026Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategiescited by 6
  2. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  3. 2025Transferring Vision-Language-Action Models to Industry Applications: Architectures, Performance, and Challengescited by 3
  4. 2024EUGens: Efficient, Unified, and General Dense Layerscited by 0
  5. 2025Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systemscited by 0
  6. 2026EUGens: Efficient, Unified, and General Dense Layerscited by 0
  7. 2026SPINE: Bridging the Cyber-Physical Gap with Agentic AIcited by 0
  8. 2026FlashDrive: Flash Vision-Language-Action Inference for Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers
  2. 2026SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Models
  3. 2026Self-Adaptive VLA for Robust Robot Deployment
  4. 2026Self-Improving VLA Policies: Selected Diffusion Noise for Spurious-Robust Action Smoothing
  5. 2026dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model
  6. 2026Patch Policy: Efficient Embodied Control via Dense Visual Representations
  7. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model
  8. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model