Soutenance de thèse Michal Muszynski
M. Michal Muszynski soutiendra en anglais, en vue de l'obtention du grade de docteur ès sciences mention informatique de la Faculté des sciences de l'Université de Genève, sa thèse intitulée:
Recognizing film aesthetics, spectators’ affect and aesthetic emotions from multimodal signals
Date: Jeudi 20 décembre 2018 à 14h00
Lieu: CUI / Battelle bâtiment A, salle de cours 404-407 (3ème étage)
- Prof. Thierry Pun (Directeur, Université de Genève)
- Dr. Guillaume Chanel (Co-Directeur, Université de Genève)
- Prof. Stéphane Marchand-Maillet (Université de Genève)
- Prof. Nadia Berthouze (University College London, United Kingdom)
- Prof. Liming Chen (Ecole Centrale de Lyon, France)
Even though aesthetic experiences are common in our lives, processes involved in aesthetic experience are not fully understood. Moreover, there is no comprehensive theory that explains and defines the concept of aesthetic experience in art. The challenge of studies on aesthetic experiences is to understand different stages of aesthetic information processing, such as perceptual analysis, cognitive processes and evaluation resulting in aesthetic judgments and emotions.
The main goal of this thesis is to analyse film aesthetic experience evoked in spectators. In particular, we aim to detect aesthetic highlights in movies, as well as recognize induced emotions and aesthetic emotions elicited in spectators. The outcomes of the research on induced emotions, aesthetic emotions and aesthetic highlights can be used for emotional and aesthetic scene detection, emotional and aesthetic scene design, video summarization and prediction of affective and aesthetic content.
In this thesis, a background review on film aesthetic experience is provided. "Everyday" and aesthetic emotions are defined and a clear distinction between induced and perceived emotions of movie audiences is made. Several emotion representations and the characterization of emotion elicitation are discussed. The concept of interpersonal synchronization with regard to watching movies together is determined. An extensive literature review on aesthetic and affective content video analysis is also provided. Existing work on aesthetic and affect recognition, as well as highlight detection from video content and spectators’ reactions is described and discussed. The main limitations of the existing state of the art research are emphasized.
Currently available aesthetic and affective multimedia databases are described in details. The continuous LIRIS-ACCEDE database that was created to study film emotional experience in a movie theater is selected and extended to study film aesthetic experience. Protocols for collecting annotations of aesthetic highlights in movies, perceived emotions and aesthetic emotions felt by movie audiences are described. The statistical analysis of the annotations is carried out.
It is shown that aesthetic highlights in movies elicit a wide range of emotions. The amount of these emotions (a level of arousal and valence intensity) strongly depends on the aesthetic highlight category and on the movie genre. Also, methodology and results of aesthetic highlight detection based on the level of synchronization among spectators’ electrodermal activity (EDA) and acceleration (ACC) measurements are presented. The results suggest that the level of synchronization among spectators’ EDA and ACC signals is discriminative for aesthetic highlight detection in the context of watching movies together. In particular, pairwise synchronization measures are stable measures of synchronization and achieve the best performance of aesthetic highlight detection independently of movie genre and highlight categories.
The relationship between induced and perceived emotions of movie audiences is investigated. An inconsistency in induced and perceived emotion annotations is observed. In particular, it is found that induced and perceived emotions of movie audiences are not always positively correlated. Furthermore, it is observed that both perceived and induced emotions are characterized by aesthetic highlights. Finally, induced emotions are recognized from spectators’ EDA and ACC measurements as well as movie content. To this end we find that Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) models outperform Support Vector Regression (SVR) and Deep Belief Network (DBN) models because their ability to take into account temporal information and hierarchically combine knowledge-inspired affective cues with audiovisual movie content and movie audience responses.
It is shown that aesthetic highlights in movies evoke aesthetic emotions in spectators that are beyond "everyday" emotions. Aesthetic emotions that are felt by spectators are associated with the category of aesthetic highlights, as well as the movie genre. In fact, movie aesthetic emotions cannot be accurately described in the arousal valence space like "everyday" emotions. Four emotional dimensions that can accurately represent aesthetic emotions are found. Furthermore, the influence of personality on aesthetic emotions is assessed by noticing the differences in aesthetic scene ratings with regard to personality traits. Also, it is shown that aesthetic emotions can be predicted based on spectators’ reactions (EDA and ACC signals).
To summarize, these promising results allow researchers to better understand processes involved in film aesthetic experience. Nevertheless, understanding of film aesthetic experience is a challenging task due to its complexity and subjective nature. Film aesthetic experience is influenced by several factors, such as personality, life experience, mood, interest that are difficult to objectively quantify. The conclusion can be made that film aesthetic experience cannot be investigated without taking into account multimodal reactions of spectators in naturalistic conditions, e.g. watching movies together in a movie theater.