# MCTensor: A High-Precision Deep Learning Library with Multi-Component Floating-Point

arXiv: https://arxiv.org/abs/2207.08867 · HTML: https://mlsanity.com/paper/2207.08867/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [Hyperbolic Graph Convolutional Neural Networks](https://mlsanity.com/paper/1910.12933/) (2019)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry](https://mlsanity.com/paper/1806.03417/) (2018)
- [Hyperbolic Graph Neural Networks](https://mlsanity.com/paper/1910.12892/) (2019)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems](https://mlsanity.com/paper/1512.01274/) (2015)
- [Training and Inference with Integers in Deep Neural Networks](https://mlsanity.com/paper/1802.04680/) (2018)
- [QPyTorch: A Low-Precision Arithmetic Simulation Framework](https://mlsanity.com/paper/1910.04540/) (2019)
- [High-Accuracy Low-Precision Training](https://mlsanity.com/paper/1803.03383/) (2018)
- [High performance ultra-low-precision convolutions on mobile devices](https://mlsanity.com/paper/1712.02427/) (2017)

## Built on by (most cited first, 3 of 3)

- [Collage: Light-Weight Low-Precision Strategy for LLM Training](https://mlsanity.com/paper/2405.03637/) (2024)
- [Low-distortion and GPU-compatible Tree Embeddings in Hyperbolic Space](https://mlsanity.com/paper/2502.17130/) (2025)
- [Rethinking the Fully Hyperbolic Vision Transformer in Polar Coordinates](https://mlsanity.com/paper/2609.32899/) (2026)

## Abstract (arXiv)

In this paper, we introduce MCTensor, a library based on PyTorch for providing general-purpose and high-precision arithmetic for DL training. MCTensor is used in the same way as PyTorch Tensor: we implement multiple basic, matrix-level computation operators and NN modules for MCTensor with identical PyTorch interface. Our algorithms achieve high precision computation and also benefits from heavily-optimized PyTorch floating-point arithmetic. We evaluate MCTensor arithmetic against PyTorch native arithmetic for a series of tasks, where models using MCTensor in float16 would match or outperform the PyTorch model with float32 or float64 precision.