# NumHTML: Numeric-Oriented Hierarchical Transformer Model for Multi-task Financial Forecasting

arXiv: https://arxiv.org/abs/2201.01770 · HTML: https://mlsanity.com/paper/2201.01770/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 51 references that are in the map.
- First posted on arXiv 5 Jan 2022.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [Pareto Multi-Task Learning](https://mlsanity.com/paper/1912.12854/) (2019)
- [Do NLP Models Know Numbers? Probing Numeracy in Embeddings](https://mlsanity.com/paper/1909.07940/) (2019)
- [Do Language Embeddings Capture Scales?](https://mlsanity.com/paper/2010.05345/) (2020)
- [Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction](https://mlsanity.com/paper/1712.02136/) (2017)
- [Modeling financial analysts' decision making via the pragmatics and semantics of earnings calls](https://mlsanity.com/paper/1906.02868/) (2019)
- [Volatility Prediction using Financial Disclosures Sentiments with Word Embedding-based IR Models](https://mlsanity.com/paper/1702.01978/) (2017)
- [Explainable Text-Driven Neural Network for Stock Prediction](https://mlsanity.com/paper/1902.04994/) (2019)

## Built on by (most cited first, 16 of 16)

- [Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models](https://mlsanity.com/paper/2402.03659/) (2024)
- [Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance](https://mlsanity.com/paper/2502.18772/) (2025)
- [Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction](https://mlsanity.com/paper/2309.00073/) (2023)
- [Causal Inference from Text: Unveiling Interactions between Variables](https://mlsanity.com/paper/2311.05286/) (2023)
- [Learning Universal Multi-level Market Irrationality Factors to Improve Stock Return Forecasting](https://mlsanity.com/paper/2502.04737/) (2025)
- [Benchmarking Classical and Quantum Models for DeFi Yield Prediction on Curve Finance](https://mlsanity.com/paper/2508.02685/) (2025)
- [Measuring Consistency in Text-based Financial Forecasting Models](https://mlsanity.com/paper/2305.08524/) (2023)
- [Predicting Corporate Risk by Jointly Modeling Company Networks and Dialogues in Earnings Conference Calls](https://mlsanity.com/paper/2206.06174/) (2022)
- [Same Company, Same Signal: The Role of Identity in Earnings Call Transcripts](https://mlsanity.com/paper/2412.18029/) (2024)
- [Enhancing Financial Sentiment Analysis with Expert-Designed Hint](https://mlsanity.com/paper/2409.17448/) (2024)
- [The Battle of Information Representations: Comparing Sentiment and Semantic Features for Forecasting Market Trends](https://mlsanity.com/paper/2303.14221/) (2023)
- [Integrating Natural Language Processing Techniques of Text Mining Into Financial System: Applications and Limitations](https://mlsanity.com/paper/2412.20438/) (2024)
- [Momentum-integrated Multi-task Stock Recommendation with Converge-based Optimization](https://mlsanity.com/paper/2509.10461/) (2025)
- [Transformer-Based Language Models Across Domain Verticals: Architectures, Applications and Critical Assessment](https://mlsanity.com/paper/2606.24331/) (2026)
- [Fast Numbers, Slow Language: Bridging Quantitative and Qualitative Earnings Signals](https://mlsanity.com/paper/2606.29734/) (2026)
- [A Shaky Voice Is Not Always a Dodge: Benchmarking Textual and Vocal Evasion Detection in Earnings Calls](https://mlsanity.com/paper/2608.28040/) (2026)

## Abstract (arXiv)

Financial forecasting has been an important and active area of machine learning research because of the challenges it presents and the potential rewards that even minor improvements in prediction accuracy or forecasting may entail. Traditionally, financial forecasting has heavily relied on quantitative indicators and metrics derived from structured financial statements. Earnings conference call data, including text and audio, is an important source of unstructured data that has been used for various prediction tasks using deep earning and related approaches. However, current deep learning-based methods are limited in the way that they deal with numeric data; numbers are typically treated as plain-text tokens without taking advantage of their underlying numeric structure. This paper describes a numeric-oriented hierarchical transformer model to predict stock returns, and financial risk using multi-modal aligned earnings calls data by taking advantage of the different categories of numbers (monetary, temporal, percentages etc.) and their magnitude. We present the results of a comprehensive evaluation of NumHTML against several state-of-the-art baselines using a real-world publicly available dataset. The results indicate that NumHTML significantly outperforms the current state-of-the-art across a variety of evaluation metrics and that it has the potential to offer significant financial gains in a practical trading context.