# Tracking objects that change in appearance with phase synchrony

arXiv: https://arxiv.org/abs/2410.02094 · HTML: https://mlsanity.com/paper/2410.02094/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 76 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 26 of 26)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [Learning long-range spatial dependencies with horizontal gated-recurrent units](https://mlsanity.com/paper/1805.08315/) (2018)
- [Robust Object Modeling for Visual Tracking](https://mlsanity.com/paper/2308.05140/) (2023)
- [Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency](https://mlsanity.com/paper/2006.16736/) (2020)
- [Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning](https://mlsanity.com/paper/2010.00763/) (2020)
- [Harmonizing the object recognition strategies of deep neural networks with humans](https://mlsanity.com/paper/2211.04533/) (2022)
- [Complex-Valued Autoencoders for Object Discovery](https://mlsanity.com/paper/2204.02075/) (2022)
- [Neuronal Synchrony in Complex-Valued Deep Networks](https://mlsanity.com/paper/1312.6115/) (2013)
- [Disentangling neural mechanisms for perceptual grouping](https://mlsanity.com/paper/1906.01558/) (2019)
- [Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex](https://mlsanity.com/paper/2306.03779/) (2023)
- [Contrastive Training of Complex-Valued Autoencoders for Object Discovery](https://mlsanity.com/paper/2305.15001/) (2023)
- [Tracking Without Re-recognition in Humans and Machines](https://mlsanity.com/paper/2105.13351/) (2021)
- [How Deep is the Feature Analysis underlying Rapid Visual Categorization?](https://mlsanity.com/paper/1606.01167/) (2016)
- [Adversarial alignment: Breaking the trade-off between the strength of an attack and its relevance to human perception](https://mlsanity.com/paper/2306.03229/) (2023)
- [KuraNet: Systems of Coupled Oscillators that Learn to Synchronize](https://mlsanity.com/paper/2105.02838/) (2021)

## Built on by (most cited first, 3 of 3)

- [Better artificial intelligence does not mean better models of biology](https://mlsanity.com/paper/2504.16940/) (2025)
- [Enhancing deep neural networks through complex-valued representations and Kuramoto synchronization dynamics](https://mlsanity.com/paper/2502.21077/) (2025)
- [GASPnet: Global Agreement to Synchronize Phases](https://mlsanity.com/paper/2507.16674/) (2025)

## Abstract (arXiv)

Objects we encounter often change appearance as we interact with them. Changes in illumination (shadows), object pose, or the movement of non-rigid objects can drastically alter available image features. How do biological visual systems track objects as they change? One plausible mechanism involves attentional mechanisms for reasoning about the locations of objects independently of their appearances -- a capability that prominent neuroscience theories have associated with computing through neural synchrony. Here, we describe a novel deep learning circuit that can learn to precisely control attention to features separately from their location in the world through neural synchrony: the complex-valued recurrent neural network (CV-RNN). Next, we compare object tracking in humans, the CV-RNN, and other deep neural networks (DNNs), using FeatureTracker: a large-scale challenge that asks observers to track objects as their locations and appearances change in precisely controlled ways. While humans effortlessly solved FeatureTracker, state-of-the-art DNNs did not. In contrast, our CV-RNN behaved similarly to humans on the challenge, providing a computational proof-of-concept for the role of phase synchronization as a neural substrate for tracking appearance-morphing objects as they move about.