Evaluation of Model Compression Techniques in NLP Tasks

This study investigates the impact of model compression techniques, particularly quantization, on various Natural Language Processing (Natural language processing, NLP) tasks using state-of-the-art models such as BERT, RoBERTa, LLaMA 3, Mistral, and Mixtral. Different quantization methods including bitsansbytes, QDQBERT, HQQ, GPTQ, and AQLM are explored to compress these models while maintaining performance across tasks. Additionally, the research evaluates the efficacy of knowledge distillation for model compression. The experiments are conducted on three different NLP tasks: Named Entity Recognition (Named Entity Recognition, NER) using the CoNLL-2003 dataset, classification using the Emotions dataset, and Extractive Question Answering (Question answering, QA) using the SQuAD2.0 dataset. Specifically, the study quantifies the performance of compressed models compared to their full-size counterparts across these tasks. The compression technique QDQBERT is applied to BERT and RoBERTa, while Mixtral and LLaMA 3 70B are compressed using AQLM. Furthermore, the study compares the performance of bitsansbytes, HQQ, and AutoGPTQ for compressing Mistral 7B and LLaMA 3 8B models. The results highlight the effectiveness of quantization methods in reducing model size and computational complexity while preserving task-specific performance metrics. This research contributes valuable insights into optimizing model efficiency for NLP tasks, addressing the growing demand for resource-efficient models in real-world applications.