Video-Based Emotional Reaction Intensity Estimation

In the present graduate qualification work the problem of estimation of intensity of emotional reactions from video materials is considered. Nowadays, the problem of emotional classification is particularly acute, which is proved by monthly worldwide competitions and publications. However, the complexity of this problem is due to the fact that most of the popular and available datasets on this topic are compiled under controlled conditions with exposed light and “played” emotions, which makes the solutions trained on such sets not so applicable in real situations with spontaneous behavior of an individual and in uncontrolled conditions. The aim of this paper is to develop an algorithm for estimating the intensity of emotional reactions based on SOTA solutions in both research and related fields. During the research, the author analyzed relevant solutions in the field of intensity estimation developed in the fifth worldwide Affective Behavior Analysis in-the-wild competition in the Emotional Reaction Intensity estimation (ERI) task. The developed solution is based on the principles of working with temporally structured data, the use of transformer-based encoders, model ensemble methods in regression problems, etc. Also within the framework of the research a prototype application was developed, which allows testing the developed model on the user's video footage.