Text Generated by Large Language Models: Detection Techniques

The rapid development and widespread adoption of large language models (LLMs) necessitate the creation of reliable and effective tools for detecting machine-generated text. Texts written using the LLMs are gramatically correct, logical and coherent, making their detection non-trivial. This work conducts a comparison study of known detection algorithms in different scenarios, including out-of-domain data. The results indicate that ensembling strengthens the robustness of the methods. A new dataset has been proposed and published, for which state-of-the-art LLMs were used: GPT4 Turbo, GPT4 Omni, Claude 3 Opus, Llama3, CommandR+, YandexGPT, GigaGhat — in total, 21000 texts were generated for the dataset. A new method for training detectors using Pairwise Ranking Loss is presented, which demonstrates higher quality compared to standard BCE fine-tuning. The source code of the experiments has been published.