mlsanityData to 14 Oct 2026

CATs++: Boosting Cost Aggregation with Convolutions and Transformers

arXiv 2202.06817 ↗First posted 14 Feb 2022cs.CVcited by 53

Not on the map. Why?

Abstract (arXiv)

Cost aggregation is a highly important process in image matching tasks, which aims to disambiguate the noisy matching scores. Existing methods generally tackle this by hand-crafted or CNN-based methods, which either lack robustness to severe deformations or inherit the limitation of CNNs that fail to discriminate incorrect matches due to limited receptive fields and inadaptability. In this paper, we introduce Cost Aggregation with Transformers (CATs) to tackle this by exploring global consensus among initial correlation map with the help of some architectural designs that allow us to fully enjoy global receptive fields of self-attention mechanism. Also, to alleviate some of the limitations that CATs may face, i.e., high computational costs induced by the use of a standard transformer that its complexity grows with the size of spatial and feature dimensions, which restrict its applicability only at limited resolution and result in rather limited performance, we propose CATs++, an extension of CATs. Our proposed methods outperform the previous state-of-the-art methods by large margins, setting a new state-of-the-art for all the benchmarks, including PF-WILLOW, PF-PASCAL, and SPair-71k. We further provide extensive ablation studies and analyses.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 82 of its 110 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

82 of 110 references.

References · 82

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2016Layer Normalizationcited by 5,669
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  11. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  12. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
38 more
  1. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  2. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  3. 2020Reformer: The Efficient Transformercited by 1,668
  4. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  5. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  6. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  7. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  9. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  10. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  11. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  12. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  13. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  14. 2021Going deeper with Image Transformerscited by 764
  15. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
  16. 2018Albumentations: fast and flexible image augmentationscited by 560
  17. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
  18. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 471
  19. 2021Early Convolutions Help Transformers See Bettercited by 396
  20. 2020TransTrack: Multiple Object Tracking with Transformercited by 371
  21. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 360
  22. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 326
  23. 2020DISK: Learning local features with policy gradientcited by 312
  24. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 288
  25. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  26. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  27. 2019R2D2: Repeatable and Reliable Detector and Descriptorcited by 207
  28. 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
  29. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 183
  30. 2018Neighbourhood Consensus Networkscited by 174
  31. 2019FSS-1000: A 1000-Class Dataset for Few-Shot Segmentationcited by 160
  32. 2019SPair-71k: A Large-scale Benchmark for Semantic Correspondencecited by 146
  33. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  34. 2017Convolutional neural network architecture for geometric matchingcited by 138
  35. 2017Visual Attribute Transfer through Deep Image Analogycited by 123
  36. 2021Learning Accurate Dense Correspondences and When to Trust Themcited by 113
  37. 2020Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutionscited by 93
  38. 2020Dual-Resolution Correspondence Networkscited by 90

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 13; 2024: 7; 2025: 19; 2026: 11.

53 papers.

Sort:

By area:

  1. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 301
  2. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 219
  3. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  4. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 136
  5. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  6. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 86
  7. 2024Local All-Pair Correspondence for Point Trackingcited by 80
  8. 2024PF3plat: Pose-Free Feed-Forward 3D Gaussian Splattingcited by 62
  9. 2024DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalizationcited by 60
  10. 2025Visual Representation Alignment for Multimodal Large Language Modelscited by 41
  11. 2023Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairscited by 37
  12. 2022Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondencecited by 32
38 more
  1. 2025Emergent Temporal Correspondences from Video Diffusion Transformerscited by 31
  2. 2024Cross-View Completion Models are Zero-shot Correspondence Estimatorscited by 30
  3. 2025Seg4Diff: Unveiling Open-Vocabulary Segmentation in Text-to-Image Diffusion Transformerscited by 24
  4. 2022MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translationcited by 22
  5. 2025Emergent Outlier View Rejection in Visual Geometry Grounded Transformerscited by 22
  6. 2023Diffusion Model for Dense Matchingcited by 21
  7. 2025C3G: Learning Compact 3D Representations with 2K Gaussianscited by 21
  8. 2023SEGIC: Unleashing the Emergent Correspondence for In-Context Segmentationcited by 19
  9. 2024Distillation of Diffusion Features for Semantic Correspondencecited by 17
  10. 2024Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondencecited by 14
  11. 2025MATCHA:Towards Matching Anythingcited by 12
  12. 2025D2USt3R: Enhancing 3D Reconstruction for Dynamic Scenescited by 10
  13. 2025SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representationscited by 7
  14. 2025Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometrycited by 7
  15. 2022Integrative Feature and Cost Aggregation with Transformers for Dense Correspondencecited by 6
  16. 2025Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondencecited by 6
  17. 2025Text-Aware Image Restoration with Diffusion Modelscited by 6
  18. 2025GECO: Geometrically Consistent Embedding with Lightspeed Inferencecited by 6
  19. 2025S^4M: Boosting Semi-Supervised Instance Segmentation with SAMcited by 5
  20. 2026PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentationcited by 5
  21. 2025Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentationcited by 4
  22. 2026TORA: Topological Representation Alignment for 3D Shape Assemblycited by 4
  23. 2023Learning Probabilistic Coordinate Fields for Robust Correspondencescited by 3
  24. 2023DocAligner: Annotating Real-world Photographic Document Images by Simply Taking Picturescited by 3
  25. 2025Semantic Correspondence: Unified Benchmarking and a Strong Baselinecited by 3
  26. 2026MARCO: Navigating the Unseen Space of Semantic Correspondencecited by 3
  27. 2023Match me if you can: Semi-Supervised Semantic Correspondence Learning with Unpaired Imagescited by 2
  28. 2026No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videoscited by 2
  29. 2026Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstructioncited by 2
  30. 2025AGILE: A Diffusion-Based Attention-Guided Image and Label Translation for Efficient Cross-Domain Plant Trait Identificationcited by 1
  31. 2025Similarity-Aware Selective State-Space Modeling for Semantic Correspondencecited by 1
  32. 2026DINO Soars: DINOv3 for Open-Vocabulary Semantic Segmentation of Remote Sensing Imagerycited by 1
  33. 2023Learning Space-Time Semantic Correspondencescited by 0
  34. 2024Matching Semantically Similar Non-Identical Objectscited by 0
  35. 2025Towards Robust Semantic Correspondence: A Benchmark and Insightscited by 0
  36. 2025Q-Align: Alleviating Attention Leakage in Zero-Shot Appearance Transfer via Query-Query Alignmentcited by 0
  37. 2026Gromov Wasserstein Optimal Transport for Semantic Correspondencescited by 0
  38. 2026PROSE: Training-Free Egocentric Scene Registration with Vision-Language Modelscited by 0

50 most cited of 53 All 53 (JSON)