# Research on Image Recognition Technology Based on Multimodal Deep Learning

arXiv: https://arxiv.org/abs/2405.03091 · HTML: https://mlsanity.com/paper/2405.03091/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 24 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 6 of 6)

- [Survival Prediction Across Diverse Cancer Types Using Neural Networks](https://mlsanity.com/paper/2404.08713/) (2024)
- [Convolutional neural network classification of cancer cytopathology images: taking breast cancer as an example](https://mlsanity.com/paper/2404.08279/) (2024)
- [Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation](https://mlsanity.com/paper/2306.04811/) (2023)
- [Improving Depth Gradient Continuity in Transformers: A Comparative Study on Monocular Depth Estimation with CNN](https://mlsanity.com/paper/2308.08333/) (2023)
- [BIMCV-R: A Landmark Dataset for 3D CT Text-Image Retrieval](https://mlsanity.com/paper/2403.15992/) (2024)
- [Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2310.04148/) (2023)

## Built on by (most cited first, 11 of 11)

- [Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers](https://mlsanity.com/paper/2406.12199/) (2024)
- [Harnessing XGBoost for Robust Biomarker Selection of Obsessive-Compulsive Disorder (OCD) from Adolescent Brain Cognitive Development (ABCD) data](https://mlsanity.com/paper/2407.00028/) (2024)
- [Automatic News Generation and Fact-Checking System Based on Language Processing](https://mlsanity.com/paper/2405.10492/) (2024)
- [Enhance Image-to-Image Generation with LLaVA-generated Prompts](https://mlsanity.com/paper/2406.01956/) (2024)
- [Image anomaly detection and prediction scheme based on SSA optimized ResNet50-BiGRU model](https://mlsanity.com/paper/2406.13987/) (2024)
- [Research on Adverse Drug Reaction Prediction Model Combining Knowledge Graph Embedding and Deep Learning](https://mlsanity.com/paper/2407.16715/) (2024)
- [Algorithm Research of ELMo Word Embedding and Deep Learning Multimodal Transformer in Image Description](https://mlsanity.com/paper/2408.06357/) (2024)
- [Research on the Application of Computer Vision Based on Deep Learning in Autonomous Driving Technology](https://mlsanity.com/paper/2406.00490/) (2024)
- [Applying Conditional Generative Adversarial Networks for Imaging Diagnosis](https://mlsanity.com/paper/2408.02074/) (2024)
- [AD-Aligning: Emulating Human-like Generalization for Cognitive Domain Adaptation in Deep Learning](https://mlsanity.com/paper/2405.09582/) (2024)
- [Online Learning of Multiple Tasks and Their Relationships : Testing on Spam Email Data and EEG Signals Recorded in Construction Fields](https://mlsanity.com/paper/2406.18311/) (2024)

## Abstract (arXiv)

This project investigates the human multi-modal behavior identification algorithm utilizing deep neural networks. According to the characteristics of different modal information, different deep neural networks are used to adapt to different modal video information. Through the integration of various deep neural networks, the algorithm successfully identifies behaviors across multiple modalities. In this project, multiple cameras developed by Microsoft Kinect were used to collect corresponding bone point data based on acquiring conventional images. In this way, the motion features in the image can be extracted. Ultimately, the behavioral characteristics discerned through both approaches are synthesized to facilitate the precise identification and categorization of behaviors. The performance of the suggested algorithm was evaluated using the MSR3D data set. The findings from these experiments indicate that the accuracy in recognizing behaviors remains consistently high, suggesting that the algorithm is reliable in various scenarios. Additionally, the tests demonstrate that the algorithm substantially enhances the accuracy of detecting pedestrian behaviors in video footage.