# Semantics in Robotics: Environmental Data Can't Yield Conventions of Human Behaviour

arXiv: https://arxiv.org/abs/2410.19308 · HTML: https://mlsanity.com/paper/2410.19308/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 97 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 15 of 15)

- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects](https://mlsanity.com/paper/1809.10790/) (2018)
- [AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection](https://mlsanity.com/paper/1709.07326/) (2017)
- [6G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communications](https://mlsanity.com/paper/2011.14844/) (2020)
- [From Semantic Communication to Semantic-aware Networking: Model, Architecture, and Open Problems](https://mlsanity.com/paper/2012.15405/) (2020)
- [The Vector Grounding Problem](https://mlsanity.com/paper/2304.01481/) (2023)
- [Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Grasping](https://mlsanity.com/paper/2011.06431/) (2020)
- [Learning Grasp Affordance Reasoning through Semantic Relations](https://mlsanity.com/paper/1906.09836/) (2019)
- [Visual Affordance and Function Understanding: A Survey](https://mlsanity.com/paper/1807.06775/) (2018)
- [Semantic-Based Explainable AI: Leveraging Semantic Scene Graphs and Pairwise Ranking to Explain Robot Failures](https://mlsanity.com/paper/2108.03554/) (2021)
- [Towards Goal-Oriented Semantic Signal Processing: Applications and Future Challenges](https://mlsanity.com/paper/2109.11885/) (2021)
- [CAGE: Context-Aware Grasping Engine](https://mlsanity.com/paper/1909.11142/) (2019)
- [Tell Me Where You Are: Multimodal LLMs Meet Place Recognition](https://mlsanity.com/paper/2406.17520/) (2024)
- [From Multi-modal Property Dataset to Robot-centric Conceptual Knowledge About Household Objects](https://mlsanity.com/paper/1906.11114/) (2019)

## Abstract (arXiv)

The word semantics, in robotics and AI, has no canonical definition. It usually serves to denote additional data provided to autonomous agents to aid HRI. Most researchers seem, implicitly, to understand that such data cannot simply be extracted from environmental data. I try to make explicit why this is so and argue that so-called semantics are best understood as data comprised of conventions of human behaviour. This includes labels, most obviously, but also places, ontologies, and affordances. Object affordances are especially problematic because they require not only semantics that are not in the environmental data (conventions of object use) but also an understanding of physics and object combinations that would, if achieved, constitute artificial superintelligence.