Accuracy and Consumption analysis from a compressed model by CompactifAI from Multiverse Computing

This study evaluates the performance of a compression method, called CompactifAI, developed by Multiverse Computing, applied to the large language model Llama 3.1 8Bcitellama. The evaluation focused on model efficiency (in terms of energy consumption) and accuracy using respectively the frameworks Codecarboncitecodecarbon and Ragasciteragas. A comparison was performed between the model compressed with CompactifAIcitecompactifaicitecompactifai2 and its full-size version. Our findings reveal that the compressed model using CompactifAI not only significantly reduced the computational resources but also maintained the model accuracy, making the model more efficient, scalable and cost-effective.