# Probing a Vision-Language-Action Model for Symbolic States and Integration into a Cognitive Architecture

arXiv: https://arxiv.org/abs/2502.04558 · HTML: https://mlsanity.com/paper/2502.04558/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning and chain-of-thought in VLA, one of 13 directions in Vision-language-action models.
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Placed in

- Vision-language-action models / [Reasoning and chain-of-thought in VLA](https://mlsanity.com/vla/reasoning-and-chain-of-thought-in-vla/)

## Builds on (most cited first, 11 of 11)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Implicit Representations of Meaning in Neural Language Models](https://mlsanity.com/paper/2106.00737/) (2021)
- [Prompting as Probing: Using Language Models for Knowledge Base Construction](https://mlsanity.com/paper/2208.11057/) (2022)
- [Probing Multimodal Embeddings for Linguistic Properties: the Visual-Semantic Case](https://mlsanity.com/paper/2102.11115/) (2021)
- [Is Bigger and Deeper Always Better? Probing LLaMA Across Scales and Layers](https://mlsanity.com/paper/2312.04333/) (2023)
- [Exploiting Language Models as a Source of Knowledge for Cognitive Agents](https://mlsanity.com/paper/2310.06846/) (2023)
- [Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making](https://mlsanity.com/paper/2408.09176/) (2024)

## Built on by (most cited first, 16 of 16)

- [Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges](https://mlsanity.com/paper/2505.04769/) (2025)
- [Pure Vision Language Action (VLA) Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.19012/) (2025)
- [Observing and Controlling Features in Vision-Language-Action Models](https://mlsanity.com/paper/2603.05487/) (2026)
- [Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models](https://mlsanity.com/paper/2603.19183/) (2026)
- [VLAs are Confined yet Capable of Generalizing to Novel Instructions](https://mlsanity.com/paper/2505.03500/) (2025)
- [An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges](https://mlsanity.com/paper/2512.11362/) (2025)
- [Decoding Task Progress from VLA Representations](https://mlsanity.com/paper/2608.13474/) (2026)
- [Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Models](https://mlsanity.com/paper/2605.00321/) (2026)
- [TEMPO: Learning Temporal Context for Dynamic Robot Manipulation](https://mlsanity.com/paper/2609.16864/) (2026)
- [ProbeAct: Probe-Guided Training-Free Failure Recovery in Vision-Language-Action Models](https://mlsanity.com/paper/2606.09740/) (2026)
- [Act on What You See: Unlocking Safe Social Navigation in Vision-Language-Action Models](https://mlsanity.com/paper/2606.10495/) (2026)
- [The Linear Representation Hypothesis for Vision-Language-Action Models](https://mlsanity.com/paper/2609.30996/) (2026)
- [Natural State-Prediction Accuracy can Hide Weak Controlled Responsiveness in VLA Readouts](https://mlsanity.com/paper/2609.34684/) (2026)
- [eRLT: Efficient VLA Reinforcement Learning via Action-Relevant Token Routing](https://mlsanity.com/paper/2610.00913/) (2026)
- [Do VLAs Understand and Adapt to the Objects They Handle, or Simply Replay Learned Behaviors?](https://mlsanity.com/paper/2610.06078/) (2026)
- [Encoded but Not in Control: Revealing the Grounding Gap in Vision-Language Robot Policies](https://mlsanity.com/paper/2610.06235/) (2026)

## Abstract (arXiv)

Vision-language-action (VLA) models hold promise as generalist robotics solutions by translating visual and linguistic inputs into robot actions, yet they lack reliability due to their black-box nature and sensitivity to environmental changes. In contrast, cognitive architectures (CA) excel in symbolic reasoning and state monitoring but are constrained by rigid predefined execution. This work bridges these approaches by probing OpenVLA's hidden layers to uncover symbolic representations of object properties, relations, and action states, enabling integration with a CA for enhanced interpretability and robustness. Through experiments on LIBERO-spatial pick-and-place tasks, we analyze the encoding of symbolic states across different layers of OpenVLA's Llama backbone. Our probing results show consistently high accuracies (> 0.90) for both object and action states across most layers, though contrary to our hypotheses, we did not observe the expected pattern of object states being encoded earlier than action states. We demonstrate an integrated DIARC-OpenVLA system that leverages these symbolic representations for real-time state monitoring, laying the foundation for more interpretable and reliable robotic manipulation.