# Triple-CFN: Separating Concepts and Features Enhances Machine Abstract Reasoning Ability

arXiv: https://arxiv.org/abs/2403.03190 · HTML: https://mlsanity.com/paper/2403.03190/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 5 Mar 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [Measuring abstract reasoning in neural networks](https://mlsanity.com/paper/1807.04225/) (2018)
- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [Attention in Natural Language Processing](https://mlsanity.com/paper/1902.02181/) (2019)
- [Deep Clustering: A Comprehensive Survey](https://mlsanity.com/paper/2210.04142/) (2022)
- [Stratified Rule-Aware Network for Abstract Visual Reasoning](https://mlsanity.com/paper/2002.06838/) (2020)
- [A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2203.04571/) (2022)
- [Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning](https://mlsanity.com/paper/2010.00763/) (2020)
- [Learning Perceptual Inference by Contrasting](https://mlsanity.com/paper/1912.00086/) (2019)
- [The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning](https://mlsanity.com/paper/2007.04212/) (2020)
- [Scale-Localized Abstract Reasoning](https://mlsanity.com/paper/2009.09405/) (2020)
- [Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution](https://mlsanity.com/paper/2103.14230/) (2021)
- [Abstract Reasoning with Distracting Features](https://mlsanity.com/paper/1912.00569/) (2019)
- [Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning](https://mlsanity.com/paper/2111.12990/) (2021)
- [Effective Abstract Reasoning with Dual-Contrast Network](https://mlsanity.com/paper/2205.13720/) (2022)
- [Solving Bongard Problems with a Visual Language and Pragmatic Reasoning](https://mlsanity.com/paper/1804.04452/) (2018)
- [Unsupervised Abstract Reasoning for Raven's Problem Matrices](https://mlsanity.com/paper/2109.10011/) (2021)
- [SAViR-T: Spatially Attentive Visual Reasoning with Transformers](https://mlsanity.com/paper/2206.09265/) (2022)
- [Multi-Viewpoint and Multi-Evaluation with Felicitous Inductive Bias Boost Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2210.14914/) (2022)
- [Abstracting Concept-Changing Rules for Solving Raven's Progressive Matrix Problems](https://mlsanity.com/paper/2307.07734/) (2023)

## Built on by (most cited first, 2 of 2)

- [Funny-Valen-Tine: Planning Solution Distribution Enhances Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2407.02688/) (2024)
- [Johnny: Structuring Representation Space to Enhance Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2506.01970/) (2025)

## Abstract (arXiv)

This paper introduces innovative frameworks for visual abstract reasoning, aiming to boost deep learning model performance. It emphasizes the importance of separating abstract concept and reasoning feature extraction processes. The effectiveness of the Cross-Feature Network (CFN) and its enhanced version, Triple-CFN, validates this approach. Challenges in visual abstract reasoning arise from complex pattern induction and conflicts in low-dimensional representations. To address these, a dual Expectation-Maximization (EM) process is introduced during CFN training, optimizing module parameters to synthesize non-conflicting concepts. However, the dual EM process may overfit, so mutual and decorrelation supervisions are designed to assist feature extraction, with decorrelation supervision proving effective. Leveraging metadata in Raven's Progressive Matrices (RPM), the paper proposes Meta Triple-CFN, improving reasoning accuracy and interpretability. Additionally, a Re-space layer is designed for feature space construction, further enhancing Triple-CFN's reasoning accuracy. These innovative designs provide effective solutions for abstract reasoning problem solvers, benefiting multiple deep learning domains. Codes are available at: https://github.com/Yuanbeiming/Triple-CFN-Separating-Concepts-and-Features-Enhances-Machine-Abstract-Reasoning-Ability.