# The Challenge of Appearance-Free Object Tracking with Feedforward Neural Networks

arXiv: https://arxiv.org/abs/2110.02772 · HTML: https://mlsanity.com/paper/2110.02772/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 7 of 7)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation](https://mlsanity.com/paper/1912.04573/) (2019)
- [Learning long-range spatial dependencies with horizontal gated-recurrent units](https://mlsanity.com/paper/1805.08315/) (2018)
- [Channel Gating Neural Networks](https://mlsanity.com/paper/1805.12549/) (2018)
- [Recurrent neural circuits for contour detection](https://mlsanity.com/paper/2010.15314/) (2020)
- [Stable and expressive recurrent vision models](https://mlsanity.com/paper/2005.11362/) (2020)
- [Tracking Noisy Targets: A Review of Recent Object Tracking Approaches](https://mlsanity.com/paper/1802.03098/) (2018)

## Built on by (most cited first, 3 of 3)

- [Extreme Image Transformations Affect Humans and Machines Differently](https://mlsanity.com/paper/2212.13967/) (2022)
- [Robustness of Humans and Machines on Object Recognition with Extreme Image Transformations](https://mlsanity.com/paper/2205.05167/) (2022)
- [Extreme Image Transformations Facilitate Robust Latent Object Representations](https://mlsanity.com/paper/2310.07725/) (2023)

## Abstract (arXiv)

Nearly all models for object tracking with artificial neural networks depend on appearance features extracted from a "backbone" architecture, designed for object recognition. Indeed, significant progress on object tracking has been spurred by introducing backbones that are better able to discriminate objects by their appearance. However, extensive neurophysiology and psychophysics evidence suggests that biological visual systems track objects using both appearance and motion features. Here, we introduce textitPathTracker, a visual challenge inspired by cognitive psychology, which tests the ability of observers to learn to track objects solely by their motion. We find that standard 3D-convolutional deep network models struggle to solve this task when clutter is introduced into the generated scenes, or when objects travel long distances. This challenge reveals that tracing the path of object motion is a blind spot of feedforward neural networks. We expect that strategies for appearance-free object tracking from biological vision can inspire solutions these failures of deep neural networks.