# SusGen-GPT: A Data-Centric LLM for Financial NLP and Sustainability Report Generation

arXiv: https://arxiv.org/abs/2412.10906 · HTML: https://mlsanity.com/paper/2412.10906/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 14 Dec 2024.

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [LoRA Learns Less and Forgets Less](https://mlsanity.com/paper/2405.09673/) (2024)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [Common 7B Language Models Already Possess Strong Math Capabilities](https://mlsanity.com/paper/2403.04706/) (2024)
- [Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis](https://mlsanity.com/paper/2305.07972/) (2023)
- [FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets](https://mlsanity.com/paper/2310.04793/) (2023)
- [Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading](https://mlsanity.com/paper/2105.12825/) (2021)
- [Financial Document Causality Detection Shared Task (FinCausal 2020)](https://mlsanity.com/paper/2012.02505/) (2020)
- [CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Tools](https://mlsanity.com/paper/2307.15770/) (2023)
- [Analyzing Sustainability Reports Using Natural Language Processing](https://mlsanity.com/paper/2011.08073/) (2020)
- [Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency](https://mlsanity.com/paper/2403.18152/) (2024)
- [ESGReveal: An LLM-based approach for extracting structured data from ESG reports](https://mlsanity.com/paper/2312.17264/) (2023)
- [Self-training Large Language Models through Knowledge Detection](https://mlsanity.com/paper/2406.11275/) (2024)

## Built on by (most cited first, 7 of 7)

- [ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge](https://mlsanity.com/paper/2506.01646/) (2025)
- [SustainableQA: A Comprehensive Question Answering Dataset for Corporate Sustainability and EU Taxonomy Reporting](https://mlsanity.com/paper/2508.03000/) (2025)
- [MMESGBench: Pioneering Multimodal Understanding and Complex Reasoning Benchmark for ESG Tasks](https://mlsanity.com/paper/2507.18932/) (2025)
- [Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning](https://mlsanity.com/paper/2502.11962/) (2025)
- [Enhancing Retrieval for ESGLLM via ESG-CID -- A Disclosure Content Index Finetuning Dataset for Mapping GRI and ESRS](https://mlsanity.com/paper/2503.10674/) (2025)
- [CSTrader: A Testbed for Language-Grounded Trading in a Community-Driven Virtual Asset Market](https://mlsanity.com/paper/2606.31461/) (2026)
- [Empirical Evaluation of Open-Source Large Language Models for Retrieval-Augmented Generation in ESG Domain](https://mlsanity.com/paper/2609.15242/) (2026)

## Abstract (arXiv)

The rapid growth of the financial sector and the rising focus on Environmental, Social, and Governance (ESG) considerations highlight the need for advanced NLP tools. However, open-source LLMs proficient in both finance and ESG domains remain scarce. To address this gap, we introduce SusGen-30K, a category-balanced dataset comprising seven financial NLP tasks and ESG report generation, and propose TCFD-Bench, a benchmark for evaluating sustainability report generation. Leveraging this dataset, we developed SusGen-GPT, a suite of models achieving state-of-the-art performance across six adapted and two off-the-shelf tasks, trailing GPT-4 by only 2% despite using 7-8B parameters compared to GPT-4's 1,700B. Based on this, we propose the SusGen system, integrated with Retrieval-Augmented Generation (RAG), to assist in sustainability report generation. This work demonstrates the efficiency of our approach, advancing research in finance and ESG.