# NEEDED: Introducing Hierarchical Transformer to Eye Diseases Diagnosis

arXiv: https://arxiv.org/abs/2212.13408 · HTML: https://mlsanity.com/paper/2212.13408/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 27 Dec 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [Hierarchical Transformers for Multi-Document Summarization](https://mlsanity.com/paper/1905.13164/) (2019)
- [HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization](https://mlsanity.com/paper/1905.06566/) (2019)
- [Hierarchical Transformers for Long Document Classification](https://mlsanity.com/paper/1910.10781/) (2019)
- [Expert Knowledge-Guided Length-Variant Hierarchical Label Generation for Proposal Classification](https://mlsanity.com/paper/2109.06661/) (2021)
- [Towards unstructured mortality prediction with free-text clinical notes](https://mlsanity.com/paper/1911.08437/) (2019)
- [Hierarchical Interdisciplinary Topic Detection Model for Research Proposal Classification](https://mlsanity.com/paper/2209.13519/) (2022)
- [Patient Risk Assessment and Warning Symptom Detection Using Deep Attention-Based Neural Networks](https://mlsanity.com/paper/1809.10804/) (2018)
- [Deep learning is effective for the classification of OCT images of normal versus Age-related Macular Degeneration](https://mlsanity.com/paper/1612.04891/) (2016)
- [Clinical Assistant Diagnosis for Electronic Medical Record Based on Convolutional Neural Network](https://mlsanity.com/paper/1804.08261/) (2018)
- [Who Should Review Your Proposal? Interdisciplinary Topic Path Detection for Research Proposals](https://mlsanity.com/paper/2203.10922/) (2022)
- [Hierarchical MixUp Multi-label Classification with Imbalanced Interdisciplinary Research Proposals](https://mlsanity.com/paper/2209.13912/) (2022)

## Built on by (most cited first, 7 of 7)

- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [Resolving the Imbalance Issue in Hierarchical Disciplinary Topic Inference via LLM-based Data Augmentation](https://mlsanity.com/paper/2310.05318/) (2023)
- [Interdisciplinary Fairness in Imbalanced Research Proposal Topic Inference: A Hierarchical Transformer-based Method with Selective Interpolation](https://mlsanity.com/paper/2309.01717/) (2023)
- [GUME: Graphs and User Modalities Enhancement for Long-Tail Multimodal Recommendation](https://mlsanity.com/paper/2407.12338/) (2024)
- [scReader: Prompting Large Language Models to Interpret scRNA-seq Data](https://mlsanity.com/paper/2412.18156/) (2024)
- [Distilling Closed-Source LLM's Knowledge for Locally Stable and Economic Biomedical Entity Linking](https://mlsanity.com/paper/2505.19722/) (2025)
- [HGT: A Hierarchical GCN-Based Transformer for Multimodal Periprosthetic Joint Infection Diagnosis Using CT Images and Text](https://mlsanity.com/paper/2305.18022/) (2023)

## Abstract (arXiv)

With the development of natural language processing techniques(NLP), automatic diagnosis of eye diseases using ophthalmology electronic medical records (OEMR) has become possible. It aims to evaluate the condition of both eyes of a patient respectively, and we formulate it as a particular multi-label classification task in this paper. Although there are a few related studies in other diseases, automatic diagnosis of eye diseases exhibits unique characteristics. First, descriptions of both eyes are mixed up in OEMR documents, with both free text and templated asymptomatic descriptions, resulting in sparsity and clutter of information. Second, OEMR documents contain multiple parts of descriptions and have long document lengths. Third, it is critical to provide explainability to the disease diagnosis model. To overcome those challenges, we present an effective automatic eye disease diagnosis framework, NEEDED. In this framework, a preprocessing module is integrated to improve the density and quality of information. Then, we design a hierarchical transformer structure for learning the contextualized representations of each sentence in the OEMR document. For the diagnosis part, we propose an attention-based predictor that enables traceable diagnosis by obtaining disease-specific information. Experiments on the real dataset and comparison with several baseline models show the advantage and explainability of our framework.