Smoothing Slot Attention Iterations and Recurrences

Slot Attention (SA) lies at the heart of mainstream Object-Centric Learning (OCL). Image features can be aggregated into object-level representations by SA textititeratively refining cold-start query slots. For video, such aggregation proceeds by SA textitrecurrently shared across frames, with queries cold-started on the first frame while transitioned from the previous frame's slots thereafter. However, cold-start queries lack sample-specific cues thus hindering precise aggregation on image or video's first frame; Non-first frames' queries are already sample-specific thus requiring aggregation transforms different from the first frame. We address these issues with our textitSmoothSA: (1) To smooth SA iterations on image or video's first frame, we textitpreheat cold-start queries with rich input-feature information, by a tiny module self-distilled inside OCL; (2) To smooth SA recurrences across video's first and non-first frames, we textitdifferentiate the homogeneous aggregation transforms by using full and single iterations respectively. Comprehensive experiments on object discovery, recognition and visual reasoning validate our method's effectiveness. Further visual analyses illuminate the underline mechanisms. Our textitsource code, textitmodel checkpoints and textittraining logs are provided on https://github.com/Genera1Z/SmoothSA.