# The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting

arXiv: https://arxiv.org/abs/2207.09295 · HTML: https://mlsanity.com/paper/2207.09295/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 149 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation](https://mlsanity.com/paper/1612.02649/) (2016)
- [Joint Discriminative and Generative Learning for Person Re-identification](https://mlsanity.com/paper/1904.07223/) (2019)
- [Scaled-YOLOv4: Scaling Cross Stage Partial Network](https://mlsanity.com/paper/2011.08036/) (2020)
- [Speed/accuracy trade-offs for modern convolutional object detectors](https://mlsanity.com/paper/1611.10012/) (2016)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Multiple Object Tracking: A Literature Review](https://mlsanity.com/paper/1409.7618/) (2014)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [TAO: A Large-Scale Benchmark for Tracking Any Object](https://mlsanity.com/paper/2005.10356/) (2020)
- [MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking](https://mlsanity.com/paper/1504.01942/) (2015)
- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)

## Built on by (most cited first, 24 of 24)

- [Align and Distill: Unifying and Improving Domain Adaptive Object Detection](https://mlsanity.com/paper/2403.12029/) (2024)
- [Smoke and Mirrors in Causal Downstream Tasks](https://mlsanity.com/paper/2405.17151/) (2024)
- [Open-World Object Counting in Videos](https://mlsanity.com/paper/2506.15368/) (2025)
- [AnuraSet: A dataset for benchmarking Neotropical anuran calls identification in passive acoustic monitoring](https://mlsanity.com/paper/2307.06860/) (2023)
- [Low-power, Continuous Remote Behavioral Localization with Event Cameras](https://mlsanity.com/paper/2312.03799/) (2023)
- [Consensus-Driven Active Model Selection](https://mlsanity.com/paper/2507.23771/) (2025)
- [Z-GMOT: Zero-shot Generic Multiple Object Tracking](https://mlsanity.com/paper/2305.17648/) (2023)
- [Insect Identification in the Wild: The AMI Dataset](https://mlsanity.com/paper/2406.12452/) (2024)
- [Active Measurement: Efficient Estimation at Scale](https://mlsanity.com/paper/2507.01372/) (2025)
- [DivShift: Exploring Domain-Specific Distribution Shifts in Large-Scale, Volunteer-Collected Biodiversity Datasets](https://mlsanity.com/paper/2410.19816/) (2024)
- [Conservation Tools: The Next Generation of Engineering--Biology Collaborations](https://mlsanity.com/paper/2301.01103/) (2023)
- [Fourier-based Action Recognition for Wildlife Behavior Quantification with Event Cameras](https://mlsanity.com/paper/2410.06698/) (2024)
- [Exploring Multimodal Foundation AI and Expert-in-the-Loop for Sustainable Management of Wild Salmon Fisheries in Indigenous Rivers](https://mlsanity.com/paper/2505.06637/) (2025)
- [When Trackers Date Fish: A Benchmark and Framework for Underwater Multiple Fish Tracking](https://mlsanity.com/paper/2507.06400/) (2025)
- [Enhanced Kalman with Adaptive Appearance Motion SORT for Grounded Generic Multiple Object Tracking](https://mlsanity.com/paper/2410.09243/) (2024)
- [SAVeD: Learning to Denoise Low-SNR Video for Improved Downstream Performance](https://mlsanity.com/paper/2504.00161/) (2025)
- [The MixCount Dataset: Bridging the Data Gap for Open-Vocabulary Object Counting](https://mlsanity.com/paper/2605.18063/) (2026)
- [OmniScientist: An Omni-Modal Omni-Discipline AI Scientist](https://mlsanity.com/paper/2608.13558/) (2026)
- [Mitigating Context Bias in Domain Adaptation for Object Detection using Mask Pooling](https://mlsanity.com/paper/2505.18446/) (2025)
- [Self-Supervised Compression and Artifact Correction for Streaming Underwater Imaging Sonar](https://mlsanity.com/paper/2511.13922/) (2025)
- [Sustainable Intelligence for the Wild: Democratizing Ecological Monitoring via Knowledge-Adaptive Edge Expert Agents](https://mlsanity.com/paper/2605.16671/) (2026)
- [Teach a Molmo2Fish: Towards interactive fish tracking with natural language guidance](https://mlsanity.com/paper/2608.18602/) (2026)
- [Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges](https://mlsanity.com/paper/2608.23845/) (2026)
- [PuTR-CouT: Counting-by-Tracking in Camera-Trap Image Sequences](https://mlsanity.com/paper/2609.05038/) (2026)

## Abstract (arXiv)

We present the Caltech Fish Counting Dataset (CFC), a large-scale dataset for detecting, tracking, and counting fish in sonar videos. We identify sonar videos as a rich source of data for advancing low signal-to-noise computer vision applications and tackling domain generalization in multiple-object tracking (MOT) and counting. In comparison to existing MOT and counting datasets, which are largely restricted to videos of people and vehicles in cities, CFC is sourced from a natural-world domain where targets are not easily resolvable and appearance features cannot be easily leveraged for target re-identification. With over half a million annotations in over 1,500 videos sourced from seven different sonar cameras, CFC allows researchers to train MOT and counting algorithms and evaluate generalization performance at unseen test locations. We perform extensive baseline experiments and identify key challenges and opportunities for advancing the state of the art in generalization in MOT and counting.