# LMC: Large Model Collaboration with Cross-assessment for Training-Free Open-Set Object Recognition

arXiv: https://arxiv.org/abs/2309.12780 · HTML: https://mlsanity.com/paper/2309.12780/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 75 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 30 of 58)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)
- [VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2303.08320/) (2023)
- [SSD: A Unified Framework for Self-Supervised Outlier Detection](https://mlsanity.com/paper/2103.12051/) (2021)
- [Black-Box Tuning for Language-Model-as-a-Service](https://mlsanity.com/paper/2201.03514/) (2022)
- [Reducing Network Agnostophobia](https://mlsanity.com/paper/1811.04110/) (2018)
- [Generative OpenMax for Multi-Class Open Set Classification](https://mlsanity.com/paper/1707.07418/) (2017)
- [Adversarial Reciprocal Points Learning for Open Set Recognition](https://mlsanity.com/paper/2103.00953/) (2021)
- [Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP](https://mlsanity.com/paper/2109.02748/) (2021)
- [Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners](https://mlsanity.com/paper/2303.02151/) (2023)
- [Classification-Reconstruction Learning for Open-Set Recognition](https://mlsanity.com/paper/1812.04246/) (2018)
- [C2AE: Class Conditioned Auto-Encoder for Open-set Recognition](https://mlsanity.com/paper/1904.01198/) (2019)
- [OpenGAN: Open-Set Recognition via Open Data Generation](https://mlsanity.com/paper/2104.02939/) (2021)
- [Robust Classification with Convolutional Prototype Learning](https://mlsanity.com/paper/1805.03438/) (2018)
- [Learning Placeholders for Open-Set Recognition](https://mlsanity.com/paper/2103.15086/) (2021)
- [Learning Open Set Network with Discriminative Reciprocal Points](https://mlsanity.com/paper/2011.00178/) (2020)

## Built on by (most cited first, 10 of 10)

- [LLMs are Good Action Recognizers](https://mlsanity.com/paper/2404.00532/) (2024)
- [LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Content](https://mlsanity.com/paper/2403.05854/) (2024)
- [A Survey on Open-Set Image Recognition](https://mlsanity.com/paper/2312.15571/) (2023)
- [Informative Sample Selection Model for Skeleton-based Action Recognition with Limited Training Samples](https://mlsanity.com/paper/2510.25345/) (2025)
- [DiffGraph: An Automated Agent-driven Model Merging Framework for In-the-Wild Text-to-Image Generation](https://mlsanity.com/paper/2603.20470/) (2026)
- [GPT-Connect: Interaction between Text-Driven Human Motion Generator and 3D Scenes in a Training-free Manner](https://mlsanity.com/paper/2403.14947/) (2024)
- [Off-the-shelf ChatGPT is a Good Few-shot Human Motion Predictor](https://mlsanity.com/paper/2405.15267/) (2024)
- [LMSeg: Unleashing the Power of Large-Scale Models for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2412.00364/) (2024)
- [CodeMerge: Codebook-Guided Model Merging for Robust Test-Time Adaptation in Autonomous Driving](https://mlsanity.com/paper/2505.16524/) (2025)
- [Training-Free Label Space Alignment for Universal Domain Adaptation](https://mlsanity.com/paper/2509.17452/) (2025)

## Abstract (arXiv)

Open-set object recognition aims to identify if an object is from a class that has been encountered during training or not. To perform open-set object recognition accurately, a key challenge is how to reduce the reliance on spurious-discriminative features. In this paper, motivated by that different large models pre-trained through different paradigms can possess very rich while distinct implicit knowledge, we propose a novel framework named Large Model Collaboration (LMC) to tackle the above challenge via collaborating different off-the-shelf large models in a training-free manner. Moreover, we also incorporate the proposed framework with several novel designs to effectively extract implicit knowledge from large models. Extensive experiments demonstrate the efficacy of our proposed framework. Code is available https://github.com/Harryqu123/LMC