mlsanityData to 14 Oct 2026

BERTer: The Efficient One

arXiv 2407.14039 ↗First posted 19 Jul 2024cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We explore advanced fine-tuning techniques to boost BERT's performance in sentiment analysis, paraphrase detection, and semantic textual similarity. Our approach leverages SMART regularization to combat overfitting, improves hyperparameter choices, employs a cross-embedding Siamese architecture for improved sentence embeddings, and introduces innovative early exiting methods. Our fine-tuning findings currently reveal substantial improvements in model efficiency and effectiveness when combining multiple fine-tuning architectures, achieving a state-of-the-art performance score of on the test set, surpassing current benchmarks and highlighting BERT's adaptability in multifaceted linguistic tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 7 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

4 of 7 references.

References · 4

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  4. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Multitask Fine-Tuning and Generative Adversarial Learning for Improved Auxiliary Classification
  2. 2024Extracting Sentence Embeddings from Pretrained Transformer Models
  3. 2024Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach
  4. 2022Extracting Latent Steering Vectors from Pretrained Language Models
  5. 2025Layer-Wise Evolution of Representations in Fine-Tuned Transformers: Insights from Sparse AutoEncoders
  6. 2025Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches
  7. 2023CoT-BERT: Enhancing Unsupervised Sentence Representation through Chain-of-Thought
  8. 2024Scaling Sparse Fine-Tuning to Large Language Models