# Augmenting BERT Carefully with Underrepresented Linguistic Features

arXiv: https://arxiv.org/abs/2011.06153 · HTML: https://mlsanity.com/paper/2011.06153/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2020.

## Built on by (most cited first, 1 of 1)

- [Comparing Acoustic-based Approaches for Alzheimer's Disease Detection](https://mlsanity.com/paper/2106.01555/) (2021)

## Abstract (arXiv)

Fine-tuned Bidirectional Encoder Representations from Transformers (BERT)-based sequence classification models have proven to be effective for detecting Alzheimer's Disease (AD) from transcripts of human speech. However, previous research shows it is possible to improve BERT's performance on various tasks by augmenting the model with additional information. In this work, we use probing tasks as introspection techniques to identify linguistic information not well-represented in various layers of BERT, but important for the AD detection task. We supplement these linguistic features in which representations from BERT are found to be insufficient with hand-crafted features externally, and show that jointly fine-tuning BERT in combination with these features improves the performance of AD classification by upto 5% over fine-tuned BERT alone.