# A Deep Neural Architecture for Harmonizing 3-D Input Data Analysis and Decision Making in Medical Imaging

arXiv: https://arxiv.org/abs/2303.00175 · HTML: https://mlsanity.com/paper/2303.00175/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 27 of 27)

- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019)
- [Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace](https://mlsanity.com/paper/1910.04855/) (2019)
- [Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond](https://mlsanity.com/paper/1804.10938/) (2018)
- [Analysing Affective Behavior in the First ABAW 2020 Competition](https://mlsanity.com/paper/2001.11409/) (2020)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study](https://mlsanity.com/paper/2105.03790/) (2021)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [Analysing Affective Behavior in the second ABAW2 Competition](https://mlsanity.com/paper/2106.15318/) (2021)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network](https://mlsanity.com/paper/1910.11111/) (2019)
- [ABAW: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.01138/) (2022)
- [COVID-CT-Dataset: A CT Scan Dataset about COVID-19](https://mlsanity.com/paper/2003.13865/) (2020)
- [Bi-Directional ConvLSTM U-Net with Densley Connected Convolutions](https://mlsanity.com/paper/1909.00166/) (2019)
- [Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition](https://mlsanity.com/paper/1811.07770/) (2018)
- [Deep Transparent Prediction through Latent Representation Analysis](https://mlsanity.com/paper/2009.07044/) (2020)
- [MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysis](https://mlsanity.com/paper/2106.07524/) (2021)
- [Exploiting multi-CNN features in CNN-RNN based Dimensional Emotion Recognition on the OMG in-the-wild Dataset](https://mlsanity.com/paper/1910.01417/) (2019)
- [Photorealistic Facial Synthesis in the Dimensional Affect Space](https://mlsanity.com/paper/1811.08004/) (2018)
- [MixAugment & Mixup: Augmentation Methods for Facial Expression Recognition](https://mlsanity.com/paper/2205.04442/) (2022)
- [A Survey on Deep Learning for Neuroimaging-based Brain Disorder Analysis](https://mlsanity.com/paper/2005.04573/) (2020)
- [A Multi-Task Learning & Generation Framework: Valence-Arousal, Action Units & Primary Expressions](https://mlsanity.com/paper/1811.07771/) (2018)
- [Training Deep Neural Networks with Different Datasets In-the-wild: The Emotion Recognition Paradigm](https://mlsanity.com/paper/1809.04359/) (2018)
- [A Multi-component CNN-RNN Approach for Dimensional Emotion Recognition in-the-wild](https://mlsanity.com/paper/1805.01452/) (2018)
- [Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans](https://mlsanity.com/paper/2101.05442/) (2021)
- [Evolutionary Multi-objective Architecture Search Framework: Application to COVID-19 3D CT Classification](https://mlsanity.com/paper/2101.10667/) (2021)

## Built on by (most cited first, 30 of 39)

- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond](https://mlsanity.com/paper/2401.01219/) (2024)
- [7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition](https://mlsanity.com/paper/2407.03835/) (2024)
- [Domain adaptation, Explainability & Fairness in AI for Medical Image Analysis: Diagnosis of COVID-19 based on 3-D Chest CT-scans](https://mlsanity.com/paper/2403.02192/) (2024)
- [COVID-19 Computer-aided Diagnosis through AI-assisted CT Imaging Analysis: Deploying a Medical AI System](https://mlsanity.com/paper/2403.06242/) (2024)
- [BTDNet: a Multi-Modal Approach for Brain Tumor Radiogenomic Classification](https://mlsanity.com/paper/2310.03485/) (2023)
- [Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkit](https://mlsanity.com/paper/2409.17717/) (2024)
- [SAM2CLIP2SAM: Vision Language Model for Segmentation of 3D CT Scans for Covid-19 Detection](https://mlsanity.com/paper/2407.15728/) (2024)
- [DVD: A Comprehensive Dataset for Advancing Violence Detection in Real-World Scenarios](https://mlsanity.com/paper/2506.05372/) (2025)
- [Robust COVID-19 Detection in CT Images with CLIP](https://mlsanity.com/paper/2403.08947/) (2024)
- [Ensuring UAV Safety: A Vision-only and Real-time Framework for Collision Avoidance Through Object Detection, Tracking, and Distance Estimation](https://mlsanity.com/paper/2405.06749/) (2024)
- [Can Machine Learning Assist in Diagnosis of Primary Immune Thrombocytopenia? A feasibility study](https://mlsanity.com/paper/2405.20562/) (2024)
- [Rethinking Affect Analysis: A Protocol for Ensuring Fairness and Consistency](https://mlsanity.com/paper/2408.02164/) (2024)
- [Using Prompts to Guide Large Language Models in Imitating a Real Person's Language Style](https://mlsanity.com/paper/2410.03848/) (2024)
- [Advancing COVID-19 Detection in 3D CT Scans](https://mlsanity.com/paper/2403.11953/) (2024)
- [Multi-Source COVID-19 Detection via Variance Risk Extrapolation](https://mlsanity.com/paper/2506.23208/) (2025)
- [Robust Facial Reactions Generation: An Emotion-Aware Framework with Modality Compensation](https://mlsanity.com/paper/2407.15798/) (2024)
- [Advancing Lung Disease Diagnosis in 3D CT Scans](https://mlsanity.com/paper/2507.00993/) (2025)
- [Domain Adaptation Using Pseudo Labels for COVID-19 Detection](https://mlsanity.com/paper/2403.11498/) (2024)
- [A Closer Look at Spatial-Slice Features Learning for COVID-19 Detection](https://mlsanity.com/paper/2404.01643/) (2024)
- [Simple 2D Convolutional Neural Network-based Approach for COVID-19 Detection](https://mlsanity.com/paper/2403.11230/) (2024)
- [Taming Domain Shift in Multi-source CT-Scan Classification via Input-Space Standardization](https://mlsanity.com/paper/2507.19858/) (2025)
- [Common Corruptions for Enhancing and Evaluating Robustness in Air-to-Air Visual Object Detection](https://mlsanity.com/paper/2405.06765/) (2024)
- [Complex Style Image Transformations for Domain Generalization in Medical Images](https://mlsanity.com/paper/2406.00298/) (2024)
- [Multi Source COVID-19 Detection via Kernel-Density-based Slice Sampling](https://mlsanity.com/paper/2507.01564/) (2025)
- [Grounding Emotion Recognition with Visual Prototypes: VEGA -- Revisiting CLIP in MERC](https://mlsanity.com/paper/2508.06564/) (2025)
- [Ensembling and Test Augmentation for Covid-19 Detection and Covid-19 Domain Adaptation from 3D CT-Scans](https://mlsanity.com/paper/2403.11338/) (2024)
- [COVID-19 detection from pulmonary CT scans using a novel EfficientNet with attention mechanism](https://mlsanity.com/paper/2403.11505/) (2024)
- [High-confidence pseudo-labels for domain adaptation in COVID-19 detection](https://mlsanity.com/paper/2403.13509/) (2024)
- [GANmut: Generating and Modifying Facial Expressions](https://mlsanity.com/paper/2406.11079/) (2024)

## Abstract (arXiv)

Harmonizing the analysis of data, especially of 3-D image volumes, consisting of different number of slices and annotated per volume, is a significant problem in training and using deep neural networks in various applications, including medical imaging. Moreover, unifying the decision making of the networks over different input datasets is crucial for the generation of rich data-driven knowledge and for trusted usage in the applications. This paper presents a new deep neural architecture, named RACNet, which includes routing and feature alignment steps and effectively handles different input lengths and single annotations of the 3-D image inputs, whilst providing highly accurate decisions. In addition, through latent variable extraction from the trained RACNet, a set of anchors are generated providing further insight on the network's decision making. These can be used to enrich and unify data-driven knowledge extracted from different datasets. An extensive experimental study illustrates the above developments, focusing on COVID-19 diagnosis through analysis of 3-D chest CT scans from databases generated in different countries and medical centers.