mlsanityData to 7 Oct 2026

SilverSight: A Multi-Task Chinese Financial Large Language Model Based on Adaptive Semantic Space Learning

arXiv 2404.04949 ↗First posted 7 Apr 2024cs.CL cs.CEcited by 1

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly being applied across various specialized fields, leveraging their extensive knowledge to empower a multitude of scenarios within these domains. However, each field encompasses a variety of specific tasks that require learning, and the diverse, heterogeneous data across these domains can lead to conflicts during model task transfer. In response to this challenge, our study introduces an Adaptive Semantic Space Learning (ASSL) framework, which utilizes the adaptive reorganization of data distributions within the semantic space to enhance the performance and selection efficacy of multi-expert models. Utilizing this framework, we trained a financial multi-task LLM named "SilverSight". Our research findings demonstrate that our framework can achieve results close to those obtained with full data training using only 10% of the data, while also exhibiting strong generalization capabilities.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 10 of its 21 references that are in the map. First posted on arXiv 7 Apr 2024.

Builds on

10 of 21 references.

References · 10

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  5. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  6. 2023Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuningcited by 148
  7. 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
  8. 2023DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuningcited by 64
  9. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  10. 2024Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMscited by 7

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.