# Efficient Multi-Task Inferencing with a Shared Backbone and Lightweight Task-Specific Adapters for Automatic Scoring

arXiv: https://arxiv.org/abs/2412.21065 · HTML: https://mlsanity.com/paper/2412.21065/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 10 references that are in the map.
- First posted on arXiv 30 Dec 2024.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2304.01933/) (2023)
- [German's Next Language Model](https://mlsanity.com/paper/2010.10906/) (2020)
- [LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuning](https://mlsanity.com/paper/2308.03303/) (2023)
- [Context Matters: A Strategy to Pre-train Language Model for Science Education](https://mlsanity.com/paper/2301.12031/) (2023)
- [Automated Scoring for Reading Comprehension via In-context BERT Tuning](https://mlsanity.com/paper/2205.09864/) (2022)
- [G-SciEdBERT: A Contextualized LLM for Science Assessment Tasks in German](https://mlsanity.com/paper/2402.06584/) (2024)

## Built on by (most cited first, 1 of 1)

- [Efficient Multi-Task Inferencing: Model Merging with Gromov-Wasserstein Feature Alignment](https://mlsanity.com/paper/2503.09774/) (2025)

## Abstract (arXiv)

The integration of Artificial Intelligence (AI) in education requires scalable and efficient frameworks that balance performance, adaptability, and cost. This paper addresses these needs by proposing a shared backbone model architecture enhanced with lightweight LoRA adapters for task-specific fine-tuning, targeting the automated scoring of student responses across 27 mutually exclusive tasks. By achieving competitive performance (average QWK of 0.848 compared to 0.888 for fully fine-tuned models) while reducing GPU memory consumption by 60% and inference latency by 40%, the framework demonstrates significant efficiency gains. This approach aligns with the workshop's focus on improving language models for educational tasks, creating responsible innovations for cost-sensitive deployment, and supporting educators by streamlining assessment workflows. The findings underscore the potential of scalable AI to enhance learning outcomes while maintaining fairness and transparency in automated scoring systems.