# AutoRC: Improving BERT Based Relation Classification Models via Architecture Search

arXiv: https://arxiv.org/abs/2009.10680 · HTML: https://mlsanity.com/paper/2009.10680/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2020.

## Built on by (most cited first, 9 of 9)

- [PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domain](https://mlsanity.com/paper/2310.14151/) (2023)
- [ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models](https://mlsanity.com/paper/2403.16187/) (2024)
- [MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning](https://mlsanity.com/paper/2410.18035/) (2024)
- [Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuning](https://mlsanity.com/paper/2203.11720/) (2022)
- [Text2MDT: Extracting Medical Decision Trees from Medical Texts](https://mlsanity.com/paper/2401.02034/) (2024)
- [IAPT: Instruction-Aware Prompt Tuning for Large Language Models](https://mlsanity.com/paper/2405.18203/) (2024)
- [Evaluating ChatGPT on Medical Information Extraction Tasks: Performance, Explainability and Beyond](https://mlsanity.com/paper/2601.21767/) (2026)
- [LV-BERT: Exploiting Layer Variety for BERT](https://mlsanity.com/paper/2106.11740/) (2021)
- [A Generative Model for Joint Multiple Intent Detection and Slot Filling](https://mlsanity.com/paper/2602.08322/) (2026)

## Abstract (arXiv)

Although BERT based relation classification (RC) models have achieved significant improvements over the traditional deep learning models, it seems that no consensus can be reached on what is the optimal architecture. Firstly, there are multiple alternatives for entity span identification. Second, there are a collection of pooling operations to aggregate the representations of entities and contexts into fixed length vectors. Third, it is difficult to manually decide which feature vectors, including their interactions, are beneficial for classifying the relation types. In this work, we design a comprehensive search space for BERT based RC models and employ neural architecture search (NAS) method to automatically discover the design choices mentioned above. Experiments on seven benchmark RC tasks show that our method is efficient and effective in finding better architectures than the baseline BERT based RC model. Ablation study demonstrates the necessity of our search space design and the effectiveness of our search method.