# OG-VLA: Orthographic Image Generation for 3D-Aware Vision-Language Action Model

arXiv: https://arxiv.org/abs/2506.01196 · HTML: https://mlsanity.com/paper/2506.01196/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in 3D and spatial-aware VLA, one of 13 directions in Vision-language-action models.
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 1 Jun 2025.

## Placed in

- Vision-language-action models / [3D and spatial-aware VLA](https://mlsanity.com/vla/3d-and-spatial-aware-vla/)

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [Language to Rewards for Robotic Skill Synthesis](https://mlsanity.com/paper/2306.08647/) (2023)
- [RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics](https://mlsanity.com/paper/2406.10721/) (2024)
- [RVT-2: Learning Precise Manipulation from Few Demonstrations](https://mlsanity.com/paper/2406.08545/) (2024)
- [VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors](https://mlsanity.com/paper/2210.11339/) (2022)
- [RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches](https://mlsanity.com/paper/2311.01977/) (2023)
- [X-VILA: Cross-Modality Alignment for Large Language Model](https://mlsanity.com/paper/2405.19335/) (2024)

## Built on by (most cited first, 13 of 13)

- [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications](https://mlsanity.com/paper/2510.07077/) (2025)
- [Pure Vision Language Action (VLA) Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.19012/) (2025)
- [VLA-0: Building State-of-the-Art VLAs with Zero Modification](https://mlsanity.com/paper/2510.13054/) (2025)
- [GLaD: Geometric Latent Distillation for Vision-Language-Action Models](https://mlsanity.com/paper/2512.09619/) (2025)
- [Flex-π: A Multi-Stream World-Action Model with Compute Flexibility](https://mlsanity.com/paper/2608.10860/) (2026)
- [Survey of Vision-Language-Action Models for Embodied Manipulation](https://mlsanity.com/paper/2508.15201/) (2025)
- [CoLA-Flow Policy: Temporally Coherent Imitation Learning via Continuous Latent Action Flow Matching for Robotic Manipulation](https://mlsanity.com/paper/2601.23087/) (2026)
- [See like a Robot: Robot-Centric Pointmaps for VLA Models](https://mlsanity.com/paper/2607.11498/) (2026)
- [BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation](https://mlsanity.com/paper/2608.05042/) (2026)
- [Cross-View Action Consistency for Camera-Robust Vision-Language-Action Policies](https://mlsanity.com/paper/2608.06965/) (2026)
- [VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation](https://mlsanity.com/paper/2607.12356/) (2026)
- [Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Models](https://mlsanity.com/paper/2608.04633/) (2026)
- [Temporal Forcing: 4D Representation Alignment for Vision-Language-Action Models](https://mlsanity.com/paper/2608.30643/) (2026)

## Abstract (arXiv)

We introduce OG-VLA, a novel architecture and learning framework that combines the generalization strengths of Vision Language Action models (VLAs) with the robustness of 3D-aware policies. We address the challenge of mapping natural language instructions and one or more RGBD observations to quasi-static robot actions. 3D-aware robot policies achieve state-of-the-art performance on precise robot manipulation tasks, but struggle with generalization to unseen instructions, scenes, and objects. On the other hand, VLAs excel at generalizing across instructions and scenes, but can be sensitive to camera and robot pose variations. We leverage prior knowledge embedded in language and vision foundation models to improve generalization of 3D-aware keyframe policies. OG-VLA unprojects input observations from diverse views into a point cloud which is then rendered from canonical orthographic views, ensuring input view invariance and consistency between input and output spaces. These canonical views are processed with a vision backbone, a Large Language Model (LLM), and an image diffusion model to generate images that encode the next position and orientation of the end-effector on the input scene. Evaluations on the Arnold and Colosseum benchmarks demonstrate state-of-the-art generalization to unseen environments, with over 40% relative improvements while maintaining robust performance in seen settings. We also show real-world adaption in 3 to 5 demonstrations along with strong generalization. Videos and resources at https://og-vla.github.io/