# Adaptation of Autoencoder for Sparsity Reduction From Clinical Notes Representation Learning

arXiv: https://arxiv.org/abs/2209.12831 · HTML: https://mlsanity.com/paper/2209.12831/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 64 references that are in the map.
- First posted on arXiv 26 Sep 2022.

## Builds on (most cited first, 11 of 11)

- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Hyper-Parameter Optimization: A Review of Algorithms and Applications](https://mlsanity.com/paper/2003.05689/) (2020)
- [On Information Plane Analyses of Neural Network Classifiers -- A Review](https://mlsanity.com/paper/2003.09671/) (2020)
- [Understanding Autoencoders with Information Theoretic Concepts](https://mlsanity.com/paper/1804.00057/) (2018)
- [Functional Regularization for Representation Learning: A Unified Theoretical Perspective](https://mlsanity.com/paper/2008.02447/) (2020)
- [Bayes Imbalance Impact Index: A Measure of Class Imbalanced Dataset for Classification Problem](https://mlsanity.com/paper/1901.10173/) (2019)
- [On the Information Plane of Autoencoders](https://mlsanity.com/paper/2005.07783/) (2020)
- [Detecting of a Patient's Condition From Clinical Narratives Using Natural Language Representation](https://mlsanity.com/paper/2104.03969/) (2021)
- [Information flows of diverse autoencoders](https://mlsanity.com/paper/2102.07402/) (2021)
- [A Critical Review of Information Bottleneck Theory and its Applications to Deep Learning](https://mlsanity.com/paper/2105.04405/) (2021)
- [Machine Learning Based on Natural Language Processing to Detect Cardiac Failure in Clinical Narratives](https://mlsanity.com/paper/2104.03934/) (2021)

## Built on by (most cited first, 4 of 4)

- [The Impact of LoRA Adapters on LLMs for Clinical Text Classification Under Computational and Data Constraints](https://mlsanity.com/paper/2407.19299/) (2024)
- [Improving Transformer Performance for French Clinical Notes Classification Using Mixture of Experts on a Limited Dataset](https://mlsanity.com/paper/2303.12892/) (2023)
- [A Novel Transformer-Based Self-Supervised Learning Method to Enhance Photoplethysmogram Signal Artifact Detection](https://mlsanity.com/paper/2401.01013/) (2024)
- [Multi-objective Representation for Numbers in Clinical Narratives: A CamemBERT-Bio-Based Alternative to Large-Scale LLMs](https://mlsanity.com/paper/2405.18448/) (2024)

## Abstract (arXiv)

When dealing with clinical text classification on a small dataset recent studies have confirmed that a well-tuned multilayer perceptron outperforms other generative classifiers, including deep learning ones. To increase the performance of the neural network classifier, feature selection for the learning representation can effectively be used. However, most feature selection methods only estimate the degree of linear dependency between variables and select the best features based on univariate statistical tests. Furthermore, the sparsity of the feature space involved in the learning representation is ignored. Goal: Our aim is therefore to access an alternative approach to tackle the sparsity by compressing the clinical representation feature space, where limited French clinical notes can also be dealt with effectively. Methods: This study proposed an autoencoder learning algorithm to take advantage of sparsity reduction in clinical note representation. The motivation was to determine how to compress sparse, high-dimensional data by reducing the dimension of the clinical note representation feature space. The classification performance of the classifiers was then evaluated in the trained and compressed feature space. Results: The proposed approach provided overall performance gains of up to 3% for each evaluation. Finally, the classifier achieved a 92% accuracy, 91% recall, 91% precision, and 91% f1-score in detecting the patient's condition. Furthermore, the compression working mechanism and the autoencoder prediction process were demonstrated by applying the theoretic information bottleneck framework.