mlsanityData to 7 Oct 2026

First Place Solution of 2023 Global Artificial Intelligence Technology Innovation Competition Track 1

arXiv 2407.01271 ↗First posted 1 Jul 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we present our champion solution to the Global Artificial Intelligence Technology Innovation Competition Track 1: Medical Imaging Diagnosis Report Generation. We select CPT-BASE as our base model for the text generation task. During the pre-training stage, we delete the mask language modeling task of CPT-BASE and instead reconstruct the vocabulary, adopting a span mask strategy and gradually increasing the number of masking ratios to perform the denoising auto-encoder pre-training task. In the fine-tuning stage, we design iterative retrieval augmentation and noise-aware similarity bucket prompt strategies. The retrieval augmentation constructs a mini-knowledge base, enriching the input information of the model, while the similarity bucket further perceives the noise information within the mini-knowledge base, guiding the model to generate higher-quality diagnostic reports based on the similarity prompts. Surprisingly, our single model has achieved a score of 2.321 on leaderboard A, and the multiple model fusion scores are 2.362 and 2.320 on the A and B leaderboards respectively, securing first place in the rankings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 34 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

16 of 34 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  3. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 216
  4. 2014Recurrent Neural Network Regularizationcited by 189
  5. 2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
  6. 2021CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generationcited by 81
  7. 2022FiD-Light: Efficient and Effective Retrieval-Augmented Text Generationcited by 63
  8. 2024Improving Open-Ended Text Generation via Adaptive Decodingcited by 21
  9. 2024Robust Semi-supervised Learning by Wisely Leveraging Open-set Datacited by 20
  10. 2024TAI++: Text as Image for Multi-Label Image Classification by Co-Learning Transferable Promptcited by 14
  11. 2023Diversify Question Generation with Retrieval-Augmented Style Transfercited by 10
  12. 2023An Invariant Learning Characterization of Controlled Text Generationcited by 9
4 more
  1. 2024Robust Semi-Supervised Learning for Self-learning Open-World Classescited by 9
  2. 2023Expository Text Generation: Imitate, Retrieve, Paraphrasecited by 8
  3. 2024MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classescited by 7
  4. 2024LSTM-Based Text Generation: A Study on Historical Datasetscited by 1

Built on by

Not cited yet.