Overcoming Template Sensitivity of Large Language Models

In the modern era of artificial intelligence, large language models stand out as extremely powerful tools for processing and analyzing textual data across various domains, from automated translation to text generation. However, despite their numerous capabilities, these models exhibit high sensitivity to the selection of input templates - pre-defined formats or structures used for crafting prompts, which can lead to significant fluctuations in quality and unreliability of results. This work examines in detail the issue of template sensitivity in large language models. We introduce a new approach that significantly reduces this sensitivity through a mix of ensemble distillation and text augmentation techniques, thereby enhancing the models' robustness. The results of the study confirm a significant improvement in model stability.