# Characterizing higher-order representations through generative diffusion models explains human decoded neurofeedback performance

arXiv: https://arxiv.org/abs/2503.14333 · HTML: https://mlsanity.com/paper/2503.14333/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 17 of its 116 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 17 of 17)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Learnable latent embeddings for joint behavioral and neural analysis](https://mlsanity.com/paper/2204.00673/) (2022)
- [Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity](https://mlsanity.com/paper/2107.04084/) (2021)
- [Explanatory models in neuroscience: Part 1 -- taking mechanistic abstraction seriously](https://mlsanity.com/paper/2104.01490/) (2021)
- [Awareness as inference in a higher-order state space](https://mlsanity.com/paper/1906.00728/) (2019)
- [Differential activation patterns in the same brain region led to opposite emotional states](https://mlsanity.com/paper/1603.01351/) (2016)
- [Statistical Neuroscience in the Single Trial Limit](https://mlsanity.com/paper/2103.05075/) (2021)
- [Studying the neural representations of uncertainty](https://mlsanity.com/paper/2202.04324/) (2022)
- [A Review of Latent Representation Models in Neuroimaging](https://mlsanity.com/paper/2412.19844/) (2024)

## Abstract (arXiv)

Brains construct not only "first-order" representations of the environment but also "higher-order" representations about those representations -- including higher-order uncertainty estimates that guide learning and adaptive behavior. Higher-order expectations about representational uncertainty -- i.e., learned through experience -- may play a key role in guiding behavior and learning, but their characterization remains empirically and theoretically challenging. Here, we introduce the Noise Estimation through Reinforcement-based Diffusion (NERD) model, a novel computational framework that trains denoising diffusion models via reinforcement learning to infer distributions of noise in functional MRI data from a decoded neurofeedback task, where healthy human participants learn to achieve target neural states. We hypothesize that participants accomplish this task by learning about and then minimizing their own representational uncertainty. We test this hypothesis with NERD, which mirrors brain-like unsupervised learning. Our results show that NERD outperforms backpropagation-trained control models in capturing human performance with explanatory power enhanced by clustering learned noise distributions. Importantly, our results also reveal individual differences in expected-uncertainty representations that predict task success, demonstrating NERD's utility as a powerful tool for probing higher-order neural representations.