Abstract

Subject of study. The development of methods for automated recognition of emotions on a face requires the creation of new stimuli sets that comprise dynamic natural emotional expressions. To date, no such databases have been designed or validated on the Russian population. Aim of study. The aims of this study were to develop a database of dynamic recordings of natural emotional facial expressions and to compare the profiles of subjective evaluation of these recordings with the results of an automated analysis of facial gestures. Method. Emotional profiles of video images of facial expressions were based on their evaluation by human observers. Study participants (N=250) rated the intensity of 33 differential emotions on a 5-point scale in each of 210 video fragments containing the natural expressions of 5 models. An automated analysis of facial structure in the video fragments of expressions was performed using OpenFace 2.0 to quantify the dynamic changes on the models’ faces. The emotional profiles of video fragments were compared with the results of automated mapping using representational similarity analysis. We calculated the rank correlation coefficient between matrices that represent the structure of subjective evaluation of expressions and their formal description. Additionally, we performed k-means cluster analysis based on subjective evaluation to identify the categorical structure of perceived emotional states. Main results. The representational similarity analysis demonstrated a significant positive correlation between subjective evaluation of expressions and their description in terms of facial actions. However, the correlation was low (0.214), which suggested a substantial variability of mimic patterns that can be subjectively perceived as similar emotions. A cluster analysis revealed five clusters corresponding to basic emotions: attention, joy, surprise, sadness, and disgust. Practical significance. The developed database of natural emotional expressions will be of interest to researchers in the field of affective computing, particularly, for the development of more effective methods for the recognition of users’ emotional states and more accurate simulation of emotional responses in robotic systems.

© 2022 Optica Publishing Group

Discriminant analysis for recognition of human face images

Kamran Etemad and Rama Chellappa
J. Opt. Soc. Am. A 14(8) 1724-1733 (1997)

Non-contact, automated cardiac pulse measurements using video imaging and blind source separation.

Ming-Zher Poh, Daniel J. McDuff, and Rosalind W. Picard
Opt. Express 18(10) 10762-10774 (2010)

Blood pressure estimation by spatial pulse-wave dynamics in a facial video

Kaito Iuchi, Ryogo Miyazaki, George C. Cardoso, Keiko Ogawa-Ochiai, and Norimichi Tsumura
Biomed. Opt. Express 13(11) 6035-6047 (2022)

Gender recognition from facial images: two or three dimensions?

Wenhao Zhang, Melvyn L. Smith, Lyndon N. Smith, and Abdul Farooq
J. Opt. Soc. Am. A 33(3) 333-344 (2016)

Automated fine structure image analysis method for discrimination of diabetic retinopathy stage using conjunctival microvasculature images

Maziyar M Khansari, William O’Neill, Richard Penn, Felix Chau, Norman P Blair, and Mahnaz Shahidi
Biomed. Opt. Express 7(7) 2597-2606 (2016)