Taras Pashchenko

In the rapidly evolving landscape of artificial intelligence, understanding the capabilities and limitations of large language models (LLMs) in specialized fields such as education is crucial. A study by Elena Kardanova, Alina Ivanova, Ksenia Tarasova, Taras Pashchenko, Aleksei Tikhoniuk, Elen Yusupova, Anatoly Kasprzhak, Yaroslav Kuzminov, Ekaterina Kruchinskaia, and Irina Brun, introduces a novel psychometrics-based methodology to assess LLM performance in the field of pedagogy. By focusing on the educational domain and developing a robust benchmark tailored for LLM evaluation, the authors offer new insights into the strengths and weaknesses of these models.

Researchers at HSE University have proposed a novel approach to assessing AI's competency in educational settings. The approach is grounded in psychometric principles and has been empirically tested using the GPT-4 model. This marks the first step in evaluating the true readiness of generative models to serve as assistants for teachers or students. The results have been published in arXiv.

From October 18 to December 17, 2022, a series of workshops dedicated to the research on the conceptual foundations in assessment and measurement of human potential took place. Within the framework of the seminars, both theoretical classes devoted to economic and sociological approaches to the study of human capital and practical workshops on the technique of its measurement were held under the guidance of the leading researchers from the Human Capital Multidisciplinary Research Center and young scientists, practitioners in human development.