# Can Transformers Smell Like Humans?

arXiv: https://arxiv.org/abs/2411.03038 · HTML: https://mlsanity.com/paper/2411.03038/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)](https://mlsanity.com/paper/1905.11833/) (2019)
- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition](https://mlsanity.com/paper/1406.3284/) (2014)
- [Brain encoding models based on multimodal transformers can transfer across language and vision](https://mlsanity.com/paper/2305.12248/) (2023)
- [Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)](https://mlsanity.com/paper/2307.10246/) (2023)

## Built on by (most cited first, 2 of 2)

- [SCENT: Aligning Mass Spectra with Molecular Structure for Olfactory Perception](https://mlsanity.com/paper/2605.27009/) (2026)
- [GraphNOSE: A Graph Transformer in Olfaction](https://mlsanity.com/paper/2609.05694/) (2026)

## Abstract (arXiv)

The human brain encodes stimuli from the environment into representations that form a sensory perception of the world. Despite recent advances in understanding visual and auditory perception, olfactory perception remains an under-explored topic in the machine learning community due to the lack of large-scale datasets annotated with labels of human olfactory perception. In this work, we ask the question of whether pre-trained transformer models of chemical structures encode representations that are aligned with human olfactory perception, i.e., can transformers smell like humans? We demonstrate that representations encoded from transformers pre-trained on general chemical structures are highly aligned with human olfactory perception. We use multiple datasets and different types of perceptual representations to show that the representations encoded by transformer models are able to predict: (i) labels associated with odorants provided by experts; (ii) continuous ratings provided by human participants with respect to pre-defined descriptors; and (iii) similarity ratings between odorants provided by human participants. Finally, we evaluate the extent to which this alignment is associated with physicochemical features of odorants known to be relevant for olfactory decoding.