# CTTA-T: Continual Test-Time Adaptation for Text Understanding via Teacher-Student with a Domain-aware and Generalized Teacher

arXiv: https://arxiv.org/abs/2512.18321 · HTML: https://mlsanity.com/paper/2512.18321/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 20 Dec 2025.

## Placed in

- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)
- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension](https://mlsanity.com/paper/1910.09753/) (2019)
- [Progressive Feature Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1811.08585/) (2018)
- [MMTEB: Massive Multilingual Text Embedding Benchmark](https://mlsanity.com/paper/2502.13595/) (2025)
- [Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factors](https://mlsanity.com/paper/2403.07366/) (2024)
- [Test-Time Model Adaptation with Only Forward Passes](https://mlsanity.com/paper/2404.01650/) (2024)
- [SoTTA: Robust Test-Time Adaptation on Noisy Data Streams](https://mlsanity.com/paper/2310.10074/) (2023)
- [Continual Test-time Domain Adaptation via Dynamic Sample Selection](https://mlsanity.com/paper/2310.03335/) (2023)
- [NoiseQA: Challenge Set Evaluation for User-Centric Question Answering](https://mlsanity.com/paper/2102.08345/) (2021)
- [Unsupervised Domain Adaptation on Reading Comprehension](https://mlsanity.com/paper/1911.06137/) (2019)
- [CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment](https://mlsanity.com/paper/2404.11932/) (2024)
- [PCoTTA: Continual Test-Time Adaptation for Multi-Task Point Cloud Understanding](https://mlsanity.com/paper/2411.00632/) (2024)
- [Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Study](https://mlsanity.com/paper/2302.04618/) (2023)
- [Test-Time Self-Adaptive Small Language Models for Question Answering](https://mlsanity.com/paper/2310.13307/) (2023)

## Built on by (most cited first, 1 of 1)

- [Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions](https://mlsanity.com/paper/2607.08164/) (2026)

## Abstract (arXiv)

Text understanding often suffers from domain shifts. To handle testing domains, domain adaptation (DA) is trained to adapt to a fixed and observed testing domain; a more challenging paradigm, test-time adaptation (TTA), cannot access the testing domain during training and online adapts to the testing samples during testing, where the samples are from a fixed domain. We aim to explore a more practical and underexplored scenario, continual test-time adaptation (CTTA) for text understanding, which involves a sequence of testing (unobserved) domains in testing. Current CTTA methods struggle in reducing error accumulation over domains and enhancing generalization to handle unobserved domains: 1) Noise-filtering reduces accumulated errors but discards useful information, and 2) accumulating historical domains enhances generalization, but it is hard to achieve adaptive accumulation. In this paper, we propose a CTTA-T (continual test-time adaptation for text understanding) framework adaptable to evolving target domains: it adopts a teacher-student framework, where the teacher is domain-aware and generalized for evolving domains. To improve teacher predictions, we propose a refine-then-filter based on dropout-driven consistency, which calibrates predictions and removes unreliable guidance. For the adaptation-generalization trade-off, we construct a domain-aware teacher by dynamically accumulating cross-domain semantics via incremental PCA, which continuously tracks domain shifts. Experiments show CTTA-T excels baselines.