An Impact of Demonstration Template Choice in In-Context Learning in Large Language Models

In recent years, Large Language Models (LLMs) have become widespread and demonstrated exceptional capabilities in adapting to various tasks through \textit{in-context learning}, which is also known as \textit{few-shot learning}. In this method, few examples are demonstrated to the model instead of detailed explanation of the task. However, the influence of the prompt template — the structure, in which examples are presented to the model — is often overlooked. This paper investigates how different prompt templates affect classification accuracy for 23 large language models, ranging from 770 million to 70 billion parameters, across 4 classification datasets, often used to evaluate new example selection methods. Our findings reveal that suboptimal template choice can drastically reduce accuracy, down to the level of random guessing in some cases. This highlights a significant flaw in the prevalent method for performance evaluation, which may give misleading results. To mitigate it, we propose a "Template Ensemble"\ approach, which aggregates predictions across multiple templates, thereby demonstrating enhanced robustness.