Using Large Language Models for Neural Architecture Search

NAS (Neural Architecture Search) is a class of methods designed to automate the search for optimal neural network architectures. This approach is enticing for the following reasons: developing the neural network architecture for a given task task is a slow, labor-intensive process that requires highly quali ed specialists and often demands considerable monetary investments. Unfortunately, classical NAS methods su er from high computational complexity and have a number of general shortcomings that impose problematic limitations on the resulting architectures. In recent years, we have witnessed an incredible rate of progress in artificial intelligence in general and Large Language Models (LLMs) in particular, largely due to the development of the Transformer architecture[17]. Large Language Models are trained on enormous datasets from the Internet, which include scienti c knowledge, papers and code. This allows them to go beyond classical natural language processing tasks and to demonstrate aptitude in mathematics, science, and coding. Experiments show that the best language models (e.g., GPT-4) can be used to select optimal neural network architectures, achieving state-of-the-art results and avoiding the main pitfalls of classical NAS methods. In this study we: 1 Develop our own method to utilise LLMs in NAS tasks. 2 Perform a comparative analysis of multiple Large Language Models for NAS. 3 Investigate whether the LLMs' behaviour in NAS is inuenced by their ability to recognize common benchmark datasets