# Diaformer: Automatic Diagnosis via Symptoms Sequence Generation

arXiv: https://arxiv.org/abs/2112.10433 · HTML: https://mlsanity.com/paper/2112.10433/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 20 Dec 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation](https://mlsanity.com/paper/2107.02137/) (2021)
- [UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training](https://mlsanity.com/paper/2002.12804/) (2020)
- [End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis](https://mlsanity.com/paper/1901.10623/) (2019)
- [End-to-End Task-Completion Neural Dialogue Systems](https://mlsanity.com/paper/1703.01008/) (2017)
- [ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation](https://mlsanity.com/paper/2001.11314/) (2020)
- [Large-Scale Object Detection in the Wild from Imbalanced Multi-Labels](https://mlsanity.com/paper/2005.08455/) (2020)
- [Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach](https://mlsanity.com/paper/2002.05513/) (2020)
- [End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.10712/) (2017)
- [Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management](https://mlsanity.com/paper/2104.04748/) (2021)
- [Prototypical Q Networks for Automatic Conversational Diagnosis and Few-Shot New Disease Adaption](https://mlsanity.com/paper/2005.11153/) (2020)

## Built on by (most cited first, 19 of 19)

- [DDXPlus: A New Dataset For Automatic Medical Diagnosis](https://mlsanity.com/paper/2205.09148/) (2022)
- [CoD, Towards an Interpretable Medical Agent using Chain of Diagnosis](https://mlsanity.com/paper/2407.13301/) (2024)
- [On Transforming Reinforcement Learning by Transformer: The Development Trajectory](https://mlsanity.com/paper/2212.14164/) (2022)
- [Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis](https://mlsanity.com/paper/2401.16107/) (2024)
- [DxFormer: A Decoupled Automatic Diagnostic System Based on Decoder-Encoder Transformer with Dense Symptom Representations](https://mlsanity.com/paper/2205.03755/) (2022)
- [Medical Dialogue Generation via Dual Flow Modeling](https://mlsanity.com/paper/2305.18109/) (2023)
- [Towards Trustworthy Automatic Diagnosis Systems by Emulating Doctors' Reasoning with Deep Reinforcement Learning](https://mlsanity.com/paper/2210.07198/) (2022)
- [CoAD: Automatic Diagnosis through Symptom and Disease Collaborative Generation](https://mlsanity.com/paper/2307.08290/) (2023)
- [DDxT: Deep Generative Transformer Models for Differential Diagnosis](https://mlsanity.com/paper/2312.01242/) (2023)
- [Generalist Foundation Models Are Not Clinical Enough for Hospital Operations](https://mlsanity.com/paper/2511.13703/) (2025)
- [Automatic Differential Diagnosis using Transformer-Based Multi-Label Sequence Classification](https://mlsanity.com/paper/2408.15827/) (2024)
- [DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaboration](https://mlsanity.com/paper/2505.18630/) (2025)
- [MIND: Unified Inquiry and Diagnosis RL with Criteria Grounded Clinical Supports for Psychiatric Consultation](https://mlsanity.com/paper/2603.03677/) (2026)
- [From Triage to Discharge: A Survey of NLP Tasks, Methods, and Open Challenges in the Emergency Department](https://mlsanity.com/paper/2608.23627/) (2026)
- [Towards Knowledge-Infused Automated Disease Diagnosis Assistant](https://mlsanity.com/paper/2405.11181/) (2024)
- [GraphMed-LT: Patient-Specific Graph Memory with Latent Clinical Thought Refinement for Multi-Turn Medical Conversations](https://mlsanity.com/paper/2510.03536/) (2025)
- [COTCAgent: Preventive Consultation via Probabilistic Chain-of-Thought Completion](https://mlsanity.com/paper/2605.15016/) (2026)
- [Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networks](https://mlsanity.com/paper/2606.05208/) (2026)
- [LLMs Anchor on Chief Complaint and Fail to Integrate Evidence in Sequential Clinical Triage](https://mlsanity.com/paper/2609.22904/) (2026)

## Abstract (arXiv)

Automatic diagnosis has attracted increasing attention but remains challenging due to multi-step reasoning. Recent works usually address it by reinforcement learning methods. However, these methods show low efficiency and require taskspecific reward functions. Considering the conversation between doctor and patient allows doctors to probe for symptoms and make diagnoses, the diagnosis process can be naturally seen as the generation of a sequence including symptoms and diagnoses. Inspired by this, we reformulate automatic diagnosis as a symptoms Sequence Generation (SG) task and propose a simple but effective automatic Diagnosis model based on Transformer (Diaformer). We firstly design the symptom attention framework to learn the generation of symptom inquiry and the disease diagnosis. To alleviate the discrepancy between sequential generation and disorder of implicit symptoms, we further design three orderless training mechanisms. Experiments on three public datasets show that our model outperforms baselines on disease diagnosis by 1%, 6% and 11.5% with the highest training efficiency. Detailed analysis on symptom inquiry prediction demonstrates that the potential of applying symptoms sequence generation for automatic diagnosis.